| [2023-03-17 10:54:05,152][24380] Saving configuration to /home/ckahmann/RL/train_dir/default_experiment/config.json... | |
| [2023-03-17 10:54:05,155][24380] Rollout worker 0 uses device cpu | |
| [2023-03-17 10:54:05,156][24380] Rollout worker 1 uses device cpu | |
| [2023-03-17 10:54:05,157][24380] Rollout worker 2 uses device cpu | |
| [2023-03-17 10:54:05,159][24380] Rollout worker 3 uses device cpu | |
| [2023-03-17 10:54:05,160][24380] Rollout worker 4 uses device cpu | |
| [2023-03-17 10:54:05,161][24380] Rollout worker 5 uses device cpu | |
| [2023-03-17 10:54:05,162][24380] Rollout worker 6 uses device cpu | |
| [2023-03-17 10:54:05,163][24380] Rollout worker 7 uses device cpu | |
| [2023-03-17 10:54:05,210][24380] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-03-17 10:54:05,211][24380] InferenceWorker_p0-w0: min num requests: 2 | |
| [2023-03-17 10:54:05,232][24380] Starting all processes... | |
| [2023-03-17 10:54:05,233][24380] Starting process learner_proc0 | |
| [2023-03-17 10:54:05,282][24380] Starting all processes... | |
| [2023-03-17 10:54:05,293][24380] Starting process inference_proc0-0 | |
| [2023-03-17 10:54:05,293][24380] Starting process rollout_proc0 | |
| [2023-03-17 10:54:05,294][24380] Starting process rollout_proc1 | |
| [2023-03-17 10:54:05,294][24380] Starting process rollout_proc2 | |
| [2023-03-17 10:54:05,294][24380] Starting process rollout_proc3 | |
| [2023-03-17 10:54:05,295][24380] Starting process rollout_proc4 | |
| [2023-03-17 10:54:05,295][24380] Starting process rollout_proc5 | |
| [2023-03-17 10:54:05,296][24380] Starting process rollout_proc6 | |
| [2023-03-17 10:54:05,297][24380] Starting process rollout_proc7 | |
| [2023-03-17 10:54:06,789][32549] Worker 0 uses CPU cores [0, 1] | |
| [2023-03-17 10:54:06,820][32535] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-03-17 10:54:06,820][32535] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 | |
| [2023-03-17 10:54:06,833][32535] Num visible devices: 1 | |
| [2023-03-17 10:54:06,847][32568] Worker 1 uses CPU cores [2, 3] | |
| [2023-03-17 10:54:06,860][32571] Worker 6 uses CPU cores [12, 13] | |
| [2023-03-17 10:54:06,880][32572] Worker 7 uses CPU cores [14, 15] | |
| [2023-03-17 10:54:06,892][32535] Starting seed is not provided | |
| [2023-03-17 10:54:06,893][32535] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-03-17 10:54:06,893][32535] Initializing actor-critic model on device cuda:0 | |
| [2023-03-17 10:54:06,893][32535] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-03-17 10:54:06,894][32535] RunningMeanStd input shape: (1,) | |
| [2023-03-17 10:54:06,904][32535] ConvEncoder: input_channels=3 | |
| [2023-03-17 10:54:06,995][32535] Conv encoder output size: 512 | |
| [2023-03-17 10:54:06,996][32535] Policy head output size: 512 | |
| [2023-03-17 10:54:07,006][32535] Created Actor Critic model with architecture: | |
| [2023-03-17 10:54:07,006][32535] ActorCriticSharedWeights( | |
| (obs_normalizer): ObservationNormalizer( | |
| (running_mean_std): RunningMeanStdDictInPlace( | |
| (running_mean_std): ModuleDict( | |
| (obs): RunningMeanStdInPlace() | |
| ) | |
| ) | |
| ) | |
| (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) | |
| (encoder): VizdoomEncoder( | |
| (basic_encoder): ConvEncoder( | |
| (enc): RecursiveScriptModule( | |
| original_name=ConvEncoderImpl | |
| (conv_head): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Conv2d) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| (2): RecursiveScriptModule(original_name=Conv2d) | |
| (3): RecursiveScriptModule(original_name=ELU) | |
| (4): RecursiveScriptModule(original_name=Conv2d) | |
| (5): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| (mlp_layers): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Linear) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| ) | |
| ) | |
| ) | |
| (core): ModelCoreRNN( | |
| (core): GRU(512, 512) | |
| ) | |
| (decoder): MlpDecoder( | |
| (mlp): Identity() | |
| ) | |
| (critic_linear): Linear(in_features=512, out_features=1, bias=True) | |
| (action_parameterization): ActionParameterizationDefault( | |
| (distribution_linear): Linear(in_features=512, out_features=5, bias=True) | |
| ) | |
| ) | |
| [2023-03-17 10:54:07,011][32570] Worker 5 uses CPU cores [10, 11] | |
| [2023-03-17 10:54:07,020][32548] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-03-17 10:54:07,020][32548] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 | |
| [2023-03-17 10:54:07,035][32567] Worker 3 uses CPU cores [6, 7] | |
| [2023-03-17 10:54:07,042][32548] Num visible devices: 1 | |
| [2023-03-17 10:54:07,165][32550] Worker 2 uses CPU cores [4, 5] | |
| [2023-03-17 10:54:07,197][32569] Worker 4 uses CPU cores [8, 9] | |
| [2023-03-17 10:54:08,475][32535] Using optimizer <class 'torch.optim.adam.Adam'> | |
| [2023-03-17 10:54:08,476][32535] No checkpoints found | |
| [2023-03-17 10:54:08,476][32535] Did not load from checkpoint, starting from scratch! | |
| [2023-03-17 10:54:08,476][32535] Initialized policy 0 weights for model version 0 | |
| [2023-03-17 10:54:08,478][32535] LearnerWorker_p0 finished initialization! | |
| [2023-03-17 10:54:08,478][32535] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-03-17 10:54:08,577][32548] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-03-17 10:54:08,578][32548] RunningMeanStd input shape: (1,) | |
| [2023-03-17 10:54:08,586][32548] ConvEncoder: input_channels=3 | |
| [2023-03-17 10:54:08,657][32548] Conv encoder output size: 512 | |
| [2023-03-17 10:54:08,657][32548] Policy head output size: 512 | |
| [2023-03-17 10:54:10,077][24380] Inference worker 0-0 is ready! | |
| [2023-03-17 10:54:10,079][24380] All inference workers are ready! Signal rollout workers to start! | |
| [2023-03-17 10:54:10,106][32549] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-03-17 10:54:10,106][32569] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-03-17 10:54:10,111][32572] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-03-17 10:54:10,111][32550] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-03-17 10:54:10,111][32571] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-03-17 10:54:10,111][32567] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-03-17 10:54:10,112][32568] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-03-17 10:54:10,112][32570] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-03-17 10:54:10,368][32569] Decorrelating experience for 0 frames... | |
| [2023-03-17 10:54:10,368][32567] Decorrelating experience for 0 frames... | |
| [2023-03-17 10:54:10,368][32550] Decorrelating experience for 0 frames... | |
| [2023-03-17 10:54:10,369][32568] Decorrelating experience for 0 frames... | |
| [2023-03-17 10:54:10,395][32549] Decorrelating experience for 0 frames... | |
| [2023-03-17 10:54:10,580][32550] Decorrelating experience for 32 frames... | |
| [2023-03-17 10:54:10,586][32567] Decorrelating experience for 32 frames... | |
| [2023-03-17 10:54:10,592][32569] Decorrelating experience for 32 frames... | |
| [2023-03-17 10:54:10,862][32568] Decorrelating experience for 32 frames... | |
| [2023-03-17 10:54:10,879][32571] Decorrelating experience for 0 frames... | |
| [2023-03-17 10:54:10,880][32567] Decorrelating experience for 64 frames... | |
| [2023-03-17 10:54:10,921][32549] Decorrelating experience for 32 frames... | |
| [2023-03-17 10:54:10,980][32569] Decorrelating experience for 64 frames... | |
| [2023-03-17 10:54:11,112][32572] Decorrelating experience for 0 frames... | |
| [2023-03-17 10:54:11,122][32571] Decorrelating experience for 32 frames... | |
| [2023-03-17 10:54:11,128][32568] Decorrelating experience for 64 frames... | |
| [2023-03-17 10:54:11,154][32550] Decorrelating experience for 64 frames... | |
| [2023-03-17 10:54:11,215][32567] Decorrelating experience for 96 frames... | |
| [2023-03-17 10:54:11,347][32569] Decorrelating experience for 96 frames... | |
| [2023-03-17 10:54:11,396][32571] Decorrelating experience for 64 frames... | |
| [2023-03-17 10:54:11,428][32568] Decorrelating experience for 96 frames... | |
| [2023-03-17 10:54:11,430][32572] Decorrelating experience for 32 frames... | |
| [2023-03-17 10:54:11,475][32550] Decorrelating experience for 96 frames... | |
| [2023-03-17 10:54:11,640][32570] Decorrelating experience for 0 frames... | |
| [2023-03-17 10:54:11,659][32549] Decorrelating experience for 64 frames... | |
| [2023-03-17 10:54:11,690][32571] Decorrelating experience for 96 frames... | |
| [2023-03-17 10:54:11,893][32570] Decorrelating experience for 32 frames... | |
| [2023-03-17 10:54:11,952][32572] Decorrelating experience for 64 frames... | |
| [2023-03-17 10:54:12,005][32549] Decorrelating experience for 96 frames... | |
| [2023-03-17 10:54:12,218][32570] Decorrelating experience for 64 frames... | |
| [2023-03-17 10:54:12,333][32572] Decorrelating experience for 96 frames... | |
| [2023-03-17 10:54:12,422][32535] Signal inference workers to stop experience collection... | |
| [2023-03-17 10:54:12,434][32548] InferenceWorker_p0-w0: stopping experience collection | |
| [2023-03-17 10:54:12,554][32570] Decorrelating experience for 96 frames... | |
| [2023-03-17 10:54:12,902][24380] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2023-03-17 10:54:12,904][24380] Avg episode reward: [(0, '2.711')] | |
| [2023-03-17 10:54:13,106][32535] Signal inference workers to resume experience collection... | |
| [2023-03-17 10:54:13,107][32548] InferenceWorker_p0-w0: resuming experience collection | |
| [2023-03-17 10:54:15,446][32548] Updated weights for policy 0, policy_version 10 (0.0263) | |
| [2023-03-17 10:54:17,635][32548] Updated weights for policy 0, policy_version 20 (0.0007) | |
| [2023-03-17 10:54:17,902][24380] Fps is (10 sec: 17203.9, 60 sec: 17203.9, 300 sec: 17203.9). Total num frames: 86016. Throughput: 0: 2777.3. Samples: 13886. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-03-17 10:54:17,903][24380] Avg episode reward: [(0, '4.460')] | |
| [2023-03-17 10:54:19,874][32548] Updated weights for policy 0, policy_version 30 (0.0008) | |
| [2023-03-17 10:54:22,150][32548] Updated weights for policy 0, policy_version 40 (0.0008) | |
| [2023-03-17 10:54:22,902][24380] Fps is (10 sec: 17612.5, 60 sec: 17612.5, 300 sec: 17612.5). Total num frames: 176128. Throughput: 0: 4135.1. Samples: 41352. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-03-17 10:54:22,904][24380] Avg episode reward: [(0, '4.468')] | |
| [2023-03-17 10:54:22,912][32535] Saving new best policy, reward=4.468! | |
| [2023-03-17 10:54:23,494][24380] Keyboard interrupt detected in the event loop EvtLoop [Runner_EvtLoop, process=main process 24380], exiting... | |
| [2023-03-17 10:54:23,497][32535] Stopping Batcher_0... | |
| [2023-03-17 10:54:23,498][32535] Loop batcher_evt_loop terminating... | |
| [2023-03-17 10:54:23,497][24380] Runner profile tree view: | |
| main_loop: 18.2648 | |
| [2023-03-17 10:54:23,499][32535] Saving /home/ckahmann/RL/train_dir/default_experiment/checkpoint_p0/checkpoint_000000045_184320.pth... | |
| [2023-03-17 10:54:23,499][24380] Collected {0: 184320}, FPS: 10091.5 | |
| [2023-03-17 10:54:23,505][32549] Stopping RolloutWorker_w0... | |
| [2023-03-17 10:54:23,506][32569] Stopping RolloutWorker_w4... | |
| [2023-03-17 10:54:23,506][32549] Loop rollout_proc0_evt_loop terminating... | |
| [2023-03-17 10:54:23,506][32569] Loop rollout_proc4_evt_loop terminating... | |
| [2023-03-17 10:54:23,509][32570] Stopping RolloutWorker_w5... | |
| [2023-03-17 10:54:23,509][32568] Stopping RolloutWorker_w1... | |
| [2023-03-17 10:54:23,510][32570] Loop rollout_proc5_evt_loop terminating... | |
| [2023-03-17 10:54:23,510][32568] Loop rollout_proc1_evt_loop terminating... | |
| [2023-03-17 10:54:23,510][32548] Weights refcount: 2 0 | |
| [2023-03-17 10:54:23,511][32548] Stopping InferenceWorker_p0-w0... | |
| [2023-03-17 10:54:23,512][32548] Loop inference_proc0-0_evt_loop terminating... | |
| [2023-03-17 10:54:23,512][32550] Stopping RolloutWorker_w2... | |
| [2023-03-17 10:54:23,513][32550] Loop rollout_proc2_evt_loop terminating... | |
| [2023-03-17 10:54:23,515][32571] Stopping RolloutWorker_w6... | |
| [2023-03-17 10:54:23,516][32571] Loop rollout_proc6_evt_loop terminating... | |
| [2023-03-17 10:54:23,516][32567] Stopping RolloutWorker_w3... | |
| [2023-03-17 10:54:23,517][32567] Loop rollout_proc3_evt_loop terminating... | |
| [2023-03-17 10:54:23,525][32572] Stopping RolloutWorker_w7... | |
| [2023-03-17 10:54:23,525][32572] Loop rollout_proc7_evt_loop terminating... | |
| [2023-03-17 10:54:23,561][32535] Stopping LearnerWorker_p0... | |
| [2023-03-17 10:54:23,562][32535] Loop learner_proc0_evt_loop terminating... | |
| [2023-03-17 10:54:29,677][24380] Environment doom_basic already registered, overwriting... | |
| [2023-03-17 10:54:29,680][24380] Environment doom_two_colors_easy already registered, overwriting... | |
| [2023-03-17 10:54:29,682][24380] Environment doom_two_colors_hard already registered, overwriting... | |
| [2023-03-17 10:54:29,684][24380] Environment doom_dm already registered, overwriting... | |
| [2023-03-17 10:54:29,686][24380] Environment doom_dwango5 already registered, overwriting... | |
| [2023-03-17 10:54:29,687][24380] Environment doom_my_way_home_flat_actions already registered, overwriting... | |
| [2023-03-17 10:54:29,689][24380] Environment doom_defend_the_center_flat_actions already registered, overwriting... | |
| [2023-03-17 10:54:29,690][24380] Environment doom_my_way_home already registered, overwriting... | |
| [2023-03-17 10:54:29,692][24380] Environment doom_deadly_corridor already registered, overwriting... | |
| [2023-03-17 10:54:29,693][24380] Environment doom_defend_the_center already registered, overwriting... | |
| [2023-03-17 10:54:29,694][24380] Environment doom_defend_the_line already registered, overwriting... | |
| [2023-03-17 10:54:29,695][24380] Environment doom_health_gathering already registered, overwriting... | |
| [2023-03-17 10:54:29,695][24380] Environment doom_health_gathering_supreme already registered, overwriting... | |
| [2023-03-17 10:54:29,696][24380] Environment doom_battle already registered, overwriting... | |
| [2023-03-17 10:54:29,698][24380] Environment doom_battle2 already registered, overwriting... | |
| [2023-03-17 10:54:29,699][24380] Environment doom_duel_bots already registered, overwriting... | |
| [2023-03-17 10:54:29,699][24380] Environment doom_deathmatch_bots already registered, overwriting... | |
| [2023-03-17 10:54:29,701][24380] Environment doom_duel already registered, overwriting... | |
| [2023-03-17 10:54:29,701][24380] Environment doom_deathmatch_full already registered, overwriting... | |
| [2023-03-17 10:54:29,702][24380] Environment doom_benchmark already registered, overwriting... | |
| [2023-03-17 10:54:29,703][24380] register_encoder_factory: <function make_vizdoom_encoder at 0x7f2990bd61f0> | |
| [2023-03-17 10:54:29,721][24380] Loading existing experiment configuration from /home/ckahmann/RL/train_dir/default_experiment/config.json | |
| [2023-03-17 10:54:29,722][24380] Overriding arg 'num_workers' with value 16 passed from command line | |
| [2023-03-17 10:54:29,729][24380] Experiment dir /home/ckahmann/RL/train_dir/default_experiment already exists! | |
| [2023-03-17 10:54:29,730][24380] Resuming existing experiment from /home/ckahmann/RL/train_dir/default_experiment... | |
| [2023-03-17 10:54:29,731][24380] Weights and Biases integration disabled | |
| [2023-03-17 10:54:29,863][24380] Environment var CUDA_VISIBLE_DEVICES is 0,1 | |
| [2023-03-17 10:54:31,128][24380] Starting experiment with the following configuration: | |
| help=False | |
| algo=APPO | |
| env=doom_health_gathering_supreme | |
| experiment=default_experiment | |
| train_dir=/home/ckahmann/RL/train_dir | |
| restart_behavior=resume | |
| device=gpu | |
| seed=None | |
| num_policies=1 | |
| async_rl=True | |
| serial_mode=False | |
| batched_sampling=False | |
| num_batches_to_accumulate=2 | |
| worker_num_splits=2 | |
| policy_workers_per_policy=1 | |
| max_policy_lag=1000 | |
| num_workers=16 | |
| num_envs_per_worker=4 | |
| batch_size=1024 | |
| num_batches_per_epoch=1 | |
| num_epochs=1 | |
| rollout=32 | |
| recurrence=32 | |
| shuffle_minibatches=False | |
| gamma=0.99 | |
| reward_scale=1.0 | |
| reward_clip=1000.0 | |
| value_bootstrap=False | |
| normalize_returns=True | |
| exploration_loss_coeff=0.001 | |
| value_loss_coeff=0.5 | |
| kl_loss_coeff=0.0 | |
| exploration_loss=symmetric_kl | |
| gae_lambda=0.95 | |
| ppo_clip_ratio=0.1 | |
| ppo_clip_value=0.2 | |
| with_vtrace=False | |
| vtrace_rho=1.0 | |
| vtrace_c=1.0 | |
| optimizer=adam | |
| adam_eps=1e-06 | |
| adam_beta1=0.9 | |
| adam_beta2=0.999 | |
| max_grad_norm=4.0 | |
| learning_rate=0.0001 | |
| lr_schedule=constant | |
| lr_schedule_kl_threshold=0.008 | |
| lr_adaptive_min=1e-06 | |
| lr_adaptive_max=0.01 | |
| obs_subtract_mean=0.0 | |
| obs_scale=255.0 | |
| normalize_input=True | |
| normalize_input_keys=None | |
| decorrelate_experience_max_seconds=0 | |
| decorrelate_envs_on_one_worker=True | |
| actor_worker_gpus=[] | |
| set_workers_cpu_affinity=True | |
| force_envs_single_thread=False | |
| default_niceness=0 | |
| log_to_file=True | |
| experiment_summaries_interval=10 | |
| flush_summaries_interval=30 | |
| stats_avg=100 | |
| summaries_use_frameskip=True | |
| heartbeat_interval=20 | |
| heartbeat_reporting_interval=600 | |
| train_for_env_steps=4000000 | |
| train_for_seconds=10000000000 | |
| save_every_sec=120 | |
| keep_checkpoints=2 | |
| load_checkpoint_kind=latest | |
| save_milestones_sec=-1 | |
| save_best_every_sec=5 | |
| save_best_metric=reward | |
| save_best_after=100000 | |
| benchmark=False | |
| encoder_mlp_layers=[512, 512] | |
| encoder_conv_architecture=convnet_simple | |
| encoder_conv_mlp_layers=[512] | |
| use_rnn=True | |
| rnn_size=512 | |
| rnn_type=gru | |
| rnn_num_layers=1 | |
| decoder_mlp_layers=[] | |
| nonlinearity=elu | |
| policy_initialization=orthogonal | |
| policy_init_gain=1.0 | |
| actor_critic_share_weights=True | |
| adaptive_stddev=True | |
| continuous_tanh_scale=0.0 | |
| initial_stddev=1.0 | |
| use_env_info_cache=False | |
| env_gpu_actions=False | |
| env_gpu_observations=True | |
| env_frameskip=4 | |
| env_framestack=1 | |
| pixel_format=CHW | |
| use_record_episode_statistics=False | |
| with_wandb=False | |
| wandb_user=None | |
| wandb_project=sample_factory | |
| wandb_group=None | |
| wandb_job_type=SF | |
| wandb_tags=[] | |
| with_pbt=False | |
| pbt_mix_policies_in_one_env=True | |
| pbt_period_env_steps=5000000 | |
| pbt_start_mutation=20000000 | |
| pbt_replace_fraction=0.3 | |
| pbt_mutation_rate=0.15 | |
| pbt_replace_reward_gap=0.1 | |
| pbt_replace_reward_gap_absolute=1e-06 | |
| pbt_optimize_gamma=False | |
| pbt_target_objective=true_objective | |
| pbt_perturb_min=1.1 | |
| pbt_perturb_max=1.5 | |
| num_agents=-1 | |
| num_humans=0 | |
| num_bots=-1 | |
| start_bot_difficulty=None | |
| timelimit=None | |
| res_w=128 | |
| res_h=72 | |
| wide_aspect_ratio=False | |
| eval_env_frameskip=1 | |
| fps=35 | |
| command_line=--env=doom_health_gathering_supreme --num_workers=8 --num_envs_per_worker=4 --train_for_env_steps=4000000 | |
| cli_args={'env': 'doom_health_gathering_supreme', 'num_workers': 8, 'num_envs_per_worker': 4, 'train_for_env_steps': 4000000} | |
| git_hash=unknown | |
| git_repo_name=not a git repository | |
| [2023-03-17 10:54:31,130][24380] Saving configuration to /home/ckahmann/RL/train_dir/default_experiment/config.json... | |
| [2023-03-17 10:54:31,132][24380] Rollout worker 0 uses device cpu | |
| [2023-03-17 10:54:31,132][24380] Rollout worker 1 uses device cpu | |
| [2023-03-17 10:54:31,133][24380] Rollout worker 2 uses device cpu | |
| [2023-03-17 10:54:31,134][24380] Rollout worker 3 uses device cpu | |
| [2023-03-17 10:54:31,134][24380] Rollout worker 4 uses device cpu | |
| [2023-03-17 10:54:31,135][24380] Rollout worker 5 uses device cpu | |
| [2023-03-17 10:54:31,136][24380] Rollout worker 6 uses device cpu | |
| [2023-03-17 10:54:31,137][24380] Rollout worker 7 uses device cpu | |
| [2023-03-17 10:54:31,137][24380] Rollout worker 8 uses device cpu | |
| [2023-03-17 10:54:31,138][24380] Rollout worker 9 uses device cpu | |
| [2023-03-17 10:54:31,139][24380] Rollout worker 10 uses device cpu | |
| [2023-03-17 10:54:31,139][24380] Rollout worker 11 uses device cpu | |
| [2023-03-17 10:54:31,140][24380] Rollout worker 12 uses device cpu | |
| [2023-03-17 10:54:31,141][24380] Rollout worker 13 uses device cpu | |
| [2023-03-17 10:54:31,141][24380] Rollout worker 14 uses device cpu | |
| [2023-03-17 10:54:31,142][24380] Rollout worker 15 uses device cpu | |
| [2023-03-17 10:54:31,206][24380] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-03-17 10:54:31,207][24380] InferenceWorker_p0-w0: min num requests: 5 | |
| [2023-03-17 10:54:31,243][24380] Starting all processes... | |
| [2023-03-17 10:54:31,244][24380] Starting process learner_proc0 | |
| [2023-03-17 10:54:31,297][24380] Starting all processes... | |
| [2023-03-17 10:54:31,305][24380] Starting process inference_proc0-0 | |
| [2023-03-17 10:54:31,306][24380] Starting process rollout_proc0 | |
| [2023-03-17 10:54:31,306][24380] Starting process rollout_proc1 | |
| [2023-03-17 10:54:31,307][24380] Starting process rollout_proc2 | |
| [2023-03-17 10:54:31,307][24380] Starting process rollout_proc3 | |
| [2023-03-17 10:54:31,307][24380] Starting process rollout_proc4 | |
| [2023-03-17 10:54:31,308][24380] Starting process rollout_proc5 | |
| [2023-03-17 10:54:31,308][24380] Starting process rollout_proc6 | |
| [2023-03-17 10:54:31,309][24380] Starting process rollout_proc7 | |
| [2023-03-17 10:54:31,310][24380] Starting process rollout_proc8 | |
| [2023-03-17 10:54:31,310][24380] Starting process rollout_proc9 | |
| [2023-03-17 10:54:31,311][24380] Starting process rollout_proc10 | |
| [2023-03-17 10:54:31,312][24380] Starting process rollout_proc11 | |
| [2023-03-17 10:54:31,312][24380] Starting process rollout_proc12 | |
| [2023-03-17 10:54:31,318][24380] Starting process rollout_proc13 | |
| [2023-03-17 10:54:31,318][24380] Starting process rollout_proc14 | |
| [2023-03-17 10:54:31,378][24380] Starting process rollout_proc15 | |
| [2023-03-17 10:54:33,386][01277] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-03-17 10:54:33,386][01277] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 | |
| [2023-03-17 10:54:33,400][01277] Num visible devices: 1 | |
| [2023-03-17 10:54:33,448][01301] Worker 1 uses CPU cores [1] | |
| [2023-03-17 10:54:33,448][01277] Starting seed is not provided | |
| [2023-03-17 10:54:33,448][01277] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-03-17 10:54:33,449][01277] Initializing actor-critic model on device cuda:0 | |
| [2023-03-17 10:54:33,449][01277] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-03-17 10:54:33,450][01277] RunningMeanStd input shape: (1,) | |
| [2023-03-17 10:54:33,466][01277] ConvEncoder: input_channels=3 | |
| [2023-03-17 10:54:33,524][01299] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-03-17 10:54:33,524][01299] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 | |
| [2023-03-17 10:54:33,536][01324] Worker 6 uses CPU cores [6] | |
| [2023-03-17 10:54:33,550][01299] Num visible devices: 1 | |
| [2023-03-17 10:54:33,552][01325] Worker 8 uses CPU cores [8] | |
| [2023-03-17 10:54:33,580][01300] Worker 0 uses CPU cores [0] | |
| [2023-03-17 10:54:33,596][01323] Worker 5 uses CPU cores [5] | |
| [2023-03-17 10:54:33,604][01321] Worker 4 uses CPU cores [4] | |
| [2023-03-17 10:54:33,648][01320] Worker 3 uses CPU cores [3] | |
| [2023-03-17 10:54:33,676][01328] Worker 10 uses CPU cores [10] | |
| [2023-03-17 10:54:33,685][01277] Conv encoder output size: 512 | |
| [2023-03-17 10:54:33,686][01277] Policy head output size: 512 | |
| [2023-03-17 10:54:33,696][01326] Worker 7 uses CPU cores [7] | |
| [2023-03-17 10:54:33,704][01277] Created Actor Critic model with architecture: | |
| [2023-03-17 10:54:33,704][01277] ActorCriticSharedWeights( | |
| (obs_normalizer): ObservationNormalizer( | |
| (running_mean_std): RunningMeanStdDictInPlace( | |
| (running_mean_std): ModuleDict( | |
| (obs): RunningMeanStdInPlace() | |
| ) | |
| ) | |
| ) | |
| (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) | |
| (encoder): VizdoomEncoder( | |
| (basic_encoder): ConvEncoder( | |
| (enc): RecursiveScriptModule( | |
| original_name=ConvEncoderImpl | |
| (conv_head): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Conv2d) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| (2): RecursiveScriptModule(original_name=Conv2d) | |
| (3): RecursiveScriptModule(original_name=ELU) | |
| (4): RecursiveScriptModule(original_name=Conv2d) | |
| (5): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| (mlp_layers): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Linear) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| ) | |
| ) | |
| ) | |
| (core): ModelCoreRNN( | |
| (core): GRU(512, 512) | |
| ) | |
| (decoder): MlpDecoder( | |
| (mlp): Identity() | |
| ) | |
| (critic_linear): Linear(in_features=512, out_features=1, bias=True) | |
| (action_parameterization): ActionParameterizationDefault( | |
| (distribution_linear): Linear(in_features=512, out_features=5, bias=True) | |
| ) | |
| ) | |
| [2023-03-17 10:54:33,752][01319] Worker 2 uses CPU cores [2] | |
| [2023-03-17 10:54:33,775][01329] Worker 11 uses CPU cores [11] | |
| [2023-03-17 10:54:33,824][01335] Worker 13 uses CPU cores [13] | |
| [2023-03-17 10:54:33,840][01332] Worker 15 uses CPU cores [15] | |
| [2023-03-17 10:54:33,876][01327] Worker 9 uses CPU cores [9] | |
| [2023-03-17 10:54:33,990][01331] Worker 12 uses CPU cores [12] | |
| [2023-03-17 10:54:33,993][01334] Worker 14 uses CPU cores [14] | |
| [2023-03-17 10:54:35,256][01277] Using optimizer <class 'torch.optim.adam.Adam'> | |
| [2023-03-17 10:54:35,257][01277] Loading state from checkpoint /home/ckahmann/RL/train_dir/default_experiment/checkpoint_p0/checkpoint_000000045_184320.pth... | |
| [2023-03-17 10:54:35,278][01277] Loading model from checkpoint | |
| [2023-03-17 10:54:35,281][01277] Loaded experiment state at self.train_step=45, self.env_steps=184320 | |
| [2023-03-17 10:54:35,281][01277] Initialized policy 0 weights for model version 45 | |
| [2023-03-17 10:54:35,283][01277] LearnerWorker_p0 finished initialization! | |
| [2023-03-17 10:54:35,283][01277] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-03-17 10:54:35,387][01299] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-03-17 10:54:35,388][01299] RunningMeanStd input shape: (1,) | |
| [2023-03-17 10:54:35,396][01299] ConvEncoder: input_channels=3 | |
| [2023-03-17 10:54:35,467][01299] Conv encoder output size: 512 | |
| [2023-03-17 10:54:35,467][01299] Policy head output size: 512 | |
| [2023-03-17 10:54:36,963][24380] Inference worker 0-0 is ready! | |
| [2023-03-17 10:54:36,965][24380] All inference workers are ready! Signal rollout workers to start! | |
| [2023-03-17 10:54:36,987][01301] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-03-17 10:54:36,987][01327] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-03-17 10:54:36,998][01323] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-03-17 10:54:36,998][01334] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-03-17 10:54:36,998][01335] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-03-17 10:54:36,998][01324] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-03-17 10:54:36,999][01326] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-03-17 10:54:36,999][01320] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-03-17 10:54:37,000][01329] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-03-17 10:54:37,000][01319] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-03-17 10:54:37,001][01321] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-03-17 10:54:37,001][01331] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-03-17 10:54:37,001][01328] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-03-17 10:54:37,001][01300] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-03-17 10:54:37,002][01325] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-03-17 10:54:37,005][01332] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-03-17 10:54:37,233][01324] Decorrelating experience for 0 frames... | |
| [2023-03-17 10:54:37,234][01319] Decorrelating experience for 0 frames... | |
| [2023-03-17 10:54:37,235][01326] Decorrelating experience for 0 frames... | |
| [2023-03-17 10:54:37,340][01301] Decorrelating experience for 0 frames... | |
| [2023-03-17 10:54:37,340][01327] Decorrelating experience for 0 frames... | |
| [2023-03-17 10:54:37,437][01319] Decorrelating experience for 32 frames... | |
| [2023-03-17 10:54:37,470][01324] Decorrelating experience for 32 frames... | |
| [2023-03-17 10:54:37,489][01335] Decorrelating experience for 0 frames... | |
| [2023-03-17 10:54:37,496][01320] Decorrelating experience for 0 frames... | |
| [2023-03-17 10:54:37,496][01321] Decorrelating experience for 0 frames... | |
| [2023-03-17 10:54:37,544][01301] Decorrelating experience for 32 frames... | |
| [2023-03-17 10:54:37,631][01334] Decorrelating experience for 0 frames... | |
| [2023-03-17 10:54:37,692][01335] Decorrelating experience for 32 frames... | |
| [2023-03-17 10:54:37,698][01321] Decorrelating experience for 32 frames... | |
| [2023-03-17 10:54:37,717][01326] Decorrelating experience for 32 frames... | |
| [2023-03-17 10:54:37,764][01327] Decorrelating experience for 32 frames... | |
| [2023-03-17 10:54:37,820][01332] Decorrelating experience for 0 frames... | |
| [2023-03-17 10:54:37,851][01325] Decorrelating experience for 0 frames... | |
| [2023-03-17 10:54:37,929][01335] Decorrelating experience for 64 frames... | |
| [2023-03-17 10:54:37,948][01328] Decorrelating experience for 0 frames... | |
| [2023-03-17 10:54:37,967][01320] Decorrelating experience for 32 frames... | |
| [2023-03-17 10:54:38,061][01301] Decorrelating experience for 64 frames... | |
| [2023-03-17 10:54:38,067][01334] Decorrelating experience for 32 frames... | |
| [2023-03-17 10:54:38,153][01321] Decorrelating experience for 64 frames... | |
| [2023-03-17 10:54:38,164][01324] Decorrelating experience for 64 frames... | |
| [2023-03-17 10:54:38,252][01326] Decorrelating experience for 64 frames... | |
| [2023-03-17 10:54:38,327][01320] Decorrelating experience for 64 frames... | |
| [2023-03-17 10:54:38,354][01332] Decorrelating experience for 32 frames... | |
| [2023-03-17 10:54:38,378][01321] Decorrelating experience for 96 frames... | |
| [2023-03-17 10:54:38,395][01319] Decorrelating experience for 64 frames... | |
| [2023-03-17 10:54:38,447][01324] Decorrelating experience for 96 frames... | |
| [2023-03-17 10:54:38,560][01334] Decorrelating experience for 64 frames... | |
| [2023-03-17 10:54:38,571][01327] Decorrelating experience for 64 frames... | |
| [2023-03-17 10:54:38,653][01335] Decorrelating experience for 96 frames... | |
| [2023-03-17 10:54:38,654][01329] Decorrelating experience for 0 frames... | |
| [2023-03-17 10:54:38,705][01332] Decorrelating experience for 64 frames... | |
| [2023-03-17 10:54:38,716][01326] Decorrelating experience for 96 frames... | |
| [2023-03-17 10:54:38,797][01301] Decorrelating experience for 96 frames... | |
| [2023-03-17 10:54:38,805][01331] Decorrelating experience for 0 frames... | |
| [2023-03-17 10:54:38,913][01300] Decorrelating experience for 0 frames... | |
| [2023-03-17 10:54:38,982][01334] Decorrelating experience for 96 frames... | |
| [2023-03-17 10:54:38,992][01332] Decorrelating experience for 96 frames... | |
| [2023-03-17 10:54:39,016][01319] Decorrelating experience for 96 frames... | |
| [2023-03-17 10:54:39,059][01331] Decorrelating experience for 32 frames... | |
| [2023-03-17 10:54:39,112][01328] Decorrelating experience for 32 frames... | |
| [2023-03-17 10:54:39,244][01323] Decorrelating experience for 0 frames... | |
| [2023-03-17 10:54:39,278][01300] Decorrelating experience for 32 frames... | |
| [2023-03-17 10:54:39,359][01329] Decorrelating experience for 32 frames... | |
| [2023-03-17 10:54:39,411][01320] Decorrelating experience for 96 frames... | |
| [2023-03-17 10:54:39,431][01331] Decorrelating experience for 64 frames... | |
| [2023-03-17 10:54:39,459][01325] Decorrelating experience for 32 frames... | |
| [2023-03-17 10:54:39,647][01300] Decorrelating experience for 64 frames... | |
| [2023-03-17 10:54:39,650][01277] Signal inference workers to stop experience collection... | |
| [2023-03-17 10:54:39,654][01299] InferenceWorker_p0-w0: stopping experience collection | |
| [2023-03-17 10:54:39,690][01323] Decorrelating experience for 32 frames... | |
| [2023-03-17 10:54:39,715][01327] Decorrelating experience for 96 frames... | |
| [2023-03-17 10:54:39,718][01328] Decorrelating experience for 64 frames... | |
| [2023-03-17 10:54:39,734][24380] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 184320. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2023-03-17 10:54:39,736][24380] Avg episode reward: [(0, '2.928')] | |
| [2023-03-17 10:54:39,808][01325] Decorrelating experience for 64 frames... | |
| [2023-03-17 10:54:39,927][01331] Decorrelating experience for 96 frames... | |
| [2023-03-17 10:54:39,930][01323] Decorrelating experience for 64 frames... | |
| [2023-03-17 10:54:39,947][01329] Decorrelating experience for 64 frames... | |
| [2023-03-17 10:54:40,043][01300] Decorrelating experience for 96 frames... | |
| [2023-03-17 10:54:40,080][01328] Decorrelating experience for 96 frames... | |
| [2023-03-17 10:54:40,145][01325] Decorrelating experience for 96 frames... | |
| [2023-03-17 10:54:40,156][01323] Decorrelating experience for 96 frames... | |
| [2023-03-17 10:54:40,287][01329] Decorrelating experience for 96 frames... | |
| [2023-03-17 10:54:40,479][01277] Signal inference workers to resume experience collection... | |
| [2023-03-17 10:54:40,480][01299] InferenceWorker_p0-w0: resuming experience collection | |
| [2023-03-17 10:54:42,279][01299] Updated weights for policy 0, policy_version 55 (0.0267) | |
| [2023-03-17 10:54:43,568][01299] Updated weights for policy 0, policy_version 65 (0.0010) | |
| [2023-03-17 10:54:44,734][24380] Fps is (10 sec: 23757.4, 60 sec: 23757.4, 300 sec: 23757.4). Total num frames: 303104. Throughput: 0: 3473.3. Samples: 17366. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0) | |
| [2023-03-17 10:54:44,735][24380] Avg episode reward: [(0, '4.797')] | |
| [2023-03-17 10:54:44,843][01299] Updated weights for policy 0, policy_version 75 (0.0011) | |
| [2023-03-17 10:54:44,884][01277] Saving new best policy, reward=4.827! | |
| [2023-03-17 10:54:46,123][01299] Updated weights for policy 0, policy_version 85 (0.0010) | |
| [2023-03-17 10:54:47,363][01299] Updated weights for policy 0, policy_version 95 (0.0012) | |
| [2023-03-17 10:54:48,642][01299] Updated weights for policy 0, policy_version 105 (0.0010) | |
| [2023-03-17 10:54:49,734][24380] Fps is (10 sec: 27853.3, 60 sec: 27853.3, 300 sec: 27853.3). Total num frames: 462848. Throughput: 0: 6522.7. Samples: 65226. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0) | |
| [2023-03-17 10:54:49,735][24380] Avg episode reward: [(0, '4.902')] | |
| [2023-03-17 10:54:49,920][01299] Updated weights for policy 0, policy_version 115 (0.0014) | |
| [2023-03-17 10:54:51,201][01299] Updated weights for policy 0, policy_version 125 (0.0012) | |
| [2023-03-17 10:54:51,201][24380] Heartbeat connected on Batcher_0 | |
| [2023-03-17 10:54:51,203][24380] Heartbeat connected on LearnerWorker_p0 | |
| [2023-03-17 10:54:51,211][24380] Heartbeat connected on InferenceWorker_p0-w0 | |
| [2023-03-17 10:54:51,212][24380] Heartbeat connected on RolloutWorker_w0 | |
| [2023-03-17 10:54:51,213][24380] Heartbeat connected on RolloutWorker_w1 | |
| [2023-03-17 10:54:51,219][24380] Heartbeat connected on RolloutWorker_w3 | |
| [2023-03-17 10:54:51,220][24380] Heartbeat connected on RolloutWorker_w2 | |
| [2023-03-17 10:54:51,222][24380] Heartbeat connected on RolloutWorker_w4 | |
| [2023-03-17 10:54:51,223][24380] Heartbeat connected on RolloutWorker_w5 | |
| [2023-03-17 10:54:51,225][24380] Heartbeat connected on RolloutWorker_w6 | |
| [2023-03-17 10:54:51,226][24380] Heartbeat connected on RolloutWorker_w7 | |
| [2023-03-17 10:54:51,232][24380] Heartbeat connected on RolloutWorker_w9 | |
| [2023-03-17 10:54:51,233][24380] Heartbeat connected on RolloutWorker_w8 | |
| [2023-03-17 10:54:51,234][24380] Heartbeat connected on RolloutWorker_w10 | |
| [2023-03-17 10:54:51,238][24380] Heartbeat connected on RolloutWorker_w12 | |
| [2023-03-17 10:54:51,239][24380] Heartbeat connected on RolloutWorker_w13 | |
| [2023-03-17 10:54:51,241][24380] Heartbeat connected on RolloutWorker_w11 | |
| [2023-03-17 10:54:51,241][24380] Heartbeat connected on RolloutWorker_w14 | |
| [2023-03-17 10:54:51,245][24380] Heartbeat connected on RolloutWorker_w15 | |
| [2023-03-17 10:54:52,537][01299] Updated weights for policy 0, policy_version 135 (0.0014) | |
| [2023-03-17 10:54:53,836][01299] Updated weights for policy 0, policy_version 145 (0.0010) | |
| [2023-03-17 10:54:54,734][24380] Fps is (10 sec: 31949.1, 60 sec: 29218.5, 300 sec: 29218.5). Total num frames: 622592. Throughput: 0: 5924.1. Samples: 88860. Policy #0 lag: (min: 0.0, avg: 1.2, max: 2.0) | |
| [2023-03-17 10:54:54,735][24380] Avg episode reward: [(0, '5.518')] | |
| [2023-03-17 10:54:54,881][01277] Saving new best policy, reward=5.635! | |
| [2023-03-17 10:54:55,142][01299] Updated weights for policy 0, policy_version 155 (0.0012) | |
| [2023-03-17 10:54:56,424][01299] Updated weights for policy 0, policy_version 165 (0.0010) | |
| [2023-03-17 10:54:57,687][01299] Updated weights for policy 0, policy_version 175 (0.0015) | |
| [2023-03-17 10:54:58,936][01299] Updated weights for policy 0, policy_version 185 (0.0009) | |
| [2023-03-17 10:54:59,734][24380] Fps is (10 sec: 31948.6, 60 sec: 29901.0, 300 sec: 29901.0). Total num frames: 782336. Throughput: 0: 6841.6. Samples: 136832. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) | |
| [2023-03-17 10:54:59,735][24380] Avg episode reward: [(0, '6.663')] | |
| [2023-03-17 10:54:59,864][01277] Saving new best policy, reward=6.669! | |
| [2023-03-17 10:55:00,226][01299] Updated weights for policy 0, policy_version 195 (0.0011) | |
| [2023-03-17 10:55:01,473][01299] Updated weights for policy 0, policy_version 205 (0.0011) | |
| [2023-03-17 10:55:02,736][01299] Updated weights for policy 0, policy_version 215 (0.0010) | |
| [2023-03-17 10:55:04,018][01299] Updated weights for policy 0, policy_version 225 (0.0013) | |
| [2023-03-17 10:55:04,734][24380] Fps is (10 sec: 31948.6, 60 sec: 30310.6, 300 sec: 30310.6). Total num frames: 942080. Throughput: 0: 7402.4. Samples: 185058. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0) | |
| [2023-03-17 10:55:04,735][24380] Avg episode reward: [(0, '7.065')] | |
| [2023-03-17 10:55:04,863][01277] Saving new best policy, reward=7.036! | |
| [2023-03-17 10:55:05,319][01299] Updated weights for policy 0, policy_version 235 (0.0009) | |
| [2023-03-17 10:55:06,589][01299] Updated weights for policy 0, policy_version 245 (0.0010) | |
| [2023-03-17 10:55:07,846][01299] Updated weights for policy 0, policy_version 255 (0.0011) | |
| [2023-03-17 10:55:09,162][01299] Updated weights for policy 0, policy_version 265 (0.0010) | |
| [2023-03-17 10:55:09,734][24380] Fps is (10 sec: 31948.6, 60 sec: 30583.5, 300 sec: 30583.5). Total num frames: 1101824. Throughput: 0: 6968.3. Samples: 209050. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0) | |
| [2023-03-17 10:55:09,736][24380] Avg episode reward: [(0, '8.389')] | |
| [2023-03-17 10:55:09,863][01277] Saving new best policy, reward=8.525! | |
| [2023-03-17 10:55:10,439][01299] Updated weights for policy 0, policy_version 275 (0.0014) | |
| [2023-03-17 10:55:11,734][01299] Updated weights for policy 0, policy_version 285 (0.0010) | |
| [2023-03-17 10:55:13,008][01299] Updated weights for policy 0, policy_version 295 (0.0014) | |
| [2023-03-17 10:55:14,302][01299] Updated weights for policy 0, policy_version 305 (0.0012) | |
| [2023-03-17 10:55:14,734][24380] Fps is (10 sec: 31948.7, 60 sec: 30778.6, 300 sec: 30778.6). Total num frames: 1261568. Throughput: 0: 7341.9. Samples: 256964. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0) | |
| [2023-03-17 10:55:14,735][24380] Avg episode reward: [(0, '10.328')] | |
| [2023-03-17 10:55:14,863][01277] Saving new best policy, reward=10.279! | |
| [2023-03-17 10:55:15,612][01299] Updated weights for policy 0, policy_version 315 (0.0011) | |
| [2023-03-17 10:55:16,888][01299] Updated weights for policy 0, policy_version 325 (0.0013) | |
| [2023-03-17 10:55:18,147][01299] Updated weights for policy 0, policy_version 335 (0.0010) | |
| [2023-03-17 10:55:19,395][01299] Updated weights for policy 0, policy_version 345 (0.0015) | |
| [2023-03-17 10:55:19,734][24380] Fps is (10 sec: 31949.0, 60 sec: 30924.9, 300 sec: 30924.9). Total num frames: 1421312. Throughput: 0: 7627.8. Samples: 305112. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) | |
| [2023-03-17 10:55:19,735][24380] Avg episode reward: [(0, '12.270')] | |
| [2023-03-17 10:55:19,863][01277] Saving new best policy, reward=12.489! | |
| [2023-03-17 10:55:20,693][01299] Updated weights for policy 0, policy_version 355 (0.0009) | |
| [2023-03-17 10:55:21,936][01299] Updated weights for policy 0, policy_version 365 (0.0013) | |
| [2023-03-17 10:55:23,234][01299] Updated weights for policy 0, policy_version 375 (0.0016) | |
| [2023-03-17 10:55:24,533][01299] Updated weights for policy 0, policy_version 385 (0.0013) | |
| [2023-03-17 10:55:24,735][24380] Fps is (10 sec: 31947.2, 60 sec: 31038.3, 300 sec: 31038.3). Total num frames: 1581056. Throughput: 0: 7309.9. Samples: 328948. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) | |
| [2023-03-17 10:55:24,736][24380] Avg episode reward: [(0, '12.939')] | |
| [2023-03-17 10:55:24,863][01277] Saving new best policy, reward=13.697! | |
| [2023-03-17 10:55:25,807][01299] Updated weights for policy 0, policy_version 395 (0.0011) | |
| [2023-03-17 10:55:27,094][01299] Updated weights for policy 0, policy_version 405 (0.0013) | |
| [2023-03-17 10:55:28,354][01299] Updated weights for policy 0, policy_version 415 (0.0010) | |
| [2023-03-17 10:55:29,639][01299] Updated weights for policy 0, policy_version 425 (0.0011) | |
| [2023-03-17 10:55:29,734][24380] Fps is (10 sec: 31949.0, 60 sec: 31129.7, 300 sec: 31129.7). Total num frames: 1740800. Throughput: 0: 7994.3. Samples: 377110. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) | |
| [2023-03-17 10:55:29,735][24380] Avg episode reward: [(0, '13.684')] | |
| [2023-03-17 10:55:30,934][01299] Updated weights for policy 0, policy_version 435 (0.0016) | |
| [2023-03-17 10:55:32,233][01299] Updated weights for policy 0, policy_version 445 (0.0011) | |
| [2023-03-17 10:55:33,489][01299] Updated weights for policy 0, policy_version 455 (0.0011) | |
| [2023-03-17 10:55:34,734][24380] Fps is (10 sec: 31950.5, 60 sec: 31204.2, 300 sec: 31204.2). Total num frames: 1900544. Throughput: 0: 8000.2. Samples: 425234. Policy #0 lag: (min: 0.0, avg: 1.4, max: 3.0) | |
| [2023-03-17 10:55:34,735][24380] Avg episode reward: [(0, '16.289')] | |
| [2023-03-17 10:55:34,772][01299] Updated weights for policy 0, policy_version 465 (0.0012) | |
| [2023-03-17 10:55:34,863][01277] Saving new best policy, reward=16.870! | |
| [2023-03-17 10:55:36,008][01299] Updated weights for policy 0, policy_version 475 (0.0013) | |
| [2023-03-17 10:55:37,281][01299] Updated weights for policy 0, policy_version 485 (0.0009) | |
| [2023-03-17 10:55:38,525][01299] Updated weights for policy 0, policy_version 495 (0.0011) | |
| [2023-03-17 10:55:39,734][24380] Fps is (10 sec: 32358.4, 60 sec: 31334.5, 300 sec: 31334.5). Total num frames: 2064384. Throughput: 0: 8013.5. Samples: 449468. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) | |
| [2023-03-17 10:55:39,735][24380] Avg episode reward: [(0, '18.133')] | |
| [2023-03-17 10:55:39,804][01299] Updated weights for policy 0, policy_version 505 (0.0010) | |
| [2023-03-17 10:55:39,863][01277] Saving new best policy, reward=17.945! | |
| [2023-03-17 10:55:41,090][01299] Updated weights for policy 0, policy_version 515 (0.0010) | |
| [2023-03-17 10:55:42,364][01299] Updated weights for policy 0, policy_version 525 (0.0011) | |
| [2023-03-17 10:55:43,627][01299] Updated weights for policy 0, policy_version 535 (0.0012) | |
| [2023-03-17 10:55:44,734][24380] Fps is (10 sec: 32358.4, 60 sec: 32017.1, 300 sec: 31381.7). Total num frames: 2224128. Throughput: 0: 8018.6. Samples: 497668. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0) | |
| [2023-03-17 10:55:44,735][24380] Avg episode reward: [(0, '17.323')] | |
| [2023-03-17 10:55:44,892][01299] Updated weights for policy 0, policy_version 545 (0.0012) | |
| [2023-03-17 10:55:46,183][01299] Updated weights for policy 0, policy_version 555 (0.0011) | |
| [2023-03-17 10:55:47,442][01299] Updated weights for policy 0, policy_version 565 (0.0015) | |
| [2023-03-17 10:55:48,670][01299] Updated weights for policy 0, policy_version 575 (0.0009) | |
| [2023-03-17 10:55:49,734][24380] Fps is (10 sec: 32358.2, 60 sec: 32085.3, 300 sec: 31480.7). Total num frames: 2387968. Throughput: 0: 8031.5. Samples: 546474. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) | |
| [2023-03-17 10:55:49,735][24380] Avg episode reward: [(0, '18.752')] | |
| [2023-03-17 10:55:49,863][01277] Saving new best policy, reward=18.918! | |
| [2023-03-17 10:55:49,969][01299] Updated weights for policy 0, policy_version 585 (0.0011) | |
| [2023-03-17 10:55:51,171][01299] Updated weights for policy 0, policy_version 595 (0.0010) | |
| [2023-03-17 10:55:52,434][01299] Updated weights for policy 0, policy_version 605 (0.0009) | |
| [2023-03-17 10:55:53,704][01299] Updated weights for policy 0, policy_version 615 (0.0012) | |
| [2023-03-17 10:55:54,734][24380] Fps is (10 sec: 32767.9, 60 sec: 32153.6, 300 sec: 31566.6). Total num frames: 2551808. Throughput: 0: 8037.4. Samples: 570730. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) | |
| [2023-03-17 10:55:54,735][24380] Avg episode reward: [(0, '17.902')] | |
| [2023-03-17 10:55:54,984][01299] Updated weights for policy 0, policy_version 625 (0.0010) | |
| [2023-03-17 10:55:56,291][01299] Updated weights for policy 0, policy_version 635 (0.0012) | |
| [2023-03-17 10:55:57,564][01299] Updated weights for policy 0, policy_version 645 (0.0013) | |
| [2023-03-17 10:55:58,843][01299] Updated weights for policy 0, policy_version 655 (0.0014) | |
| [2023-03-17 10:55:59,734][24380] Fps is (10 sec: 31949.0, 60 sec: 32085.4, 300 sec: 31539.3). Total num frames: 2707456. Throughput: 0: 8041.9. Samples: 618848. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) | |
| [2023-03-17 10:55:59,735][24380] Avg episode reward: [(0, '19.811')] | |
| [2023-03-17 10:55:59,863][01277] Saving new best policy, reward=19.975! | |
| [2023-03-17 10:56:00,116][01299] Updated weights for policy 0, policy_version 665 (0.0015) | |
| [2023-03-17 10:56:01,390][01299] Updated weights for policy 0, policy_version 675 (0.0010) | |
| [2023-03-17 10:56:02,654][01299] Updated weights for policy 0, policy_version 685 (0.0012) | |
| [2023-03-17 10:56:03,932][01299] Updated weights for policy 0, policy_version 695 (0.0012) | |
| [2023-03-17 10:56:04,734][24380] Fps is (10 sec: 31949.0, 60 sec: 32153.6, 300 sec: 31611.6). Total num frames: 2871296. Throughput: 0: 8045.4. Samples: 667156. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) | |
| [2023-03-17 10:56:04,735][24380] Avg episode reward: [(0, '20.363')] | |
| [2023-03-17 10:56:04,863][01277] Saving new best policy, reward=20.373! | |
| [2023-03-17 10:56:05,206][01299] Updated weights for policy 0, policy_version 705 (0.0009) | |
| [2023-03-17 10:56:06,462][01299] Updated weights for policy 0, policy_version 715 (0.0012) | |
| [2023-03-17 10:56:07,751][01299] Updated weights for policy 0, policy_version 725 (0.0013) | |
| [2023-03-17 10:56:09,058][01299] Updated weights for policy 0, policy_version 735 (0.0011) | |
| [2023-03-17 10:56:09,734][24380] Fps is (10 sec: 32358.4, 60 sec: 32153.7, 300 sec: 31630.3). Total num frames: 3031040. Throughput: 0: 8050.5. Samples: 691216. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) | |
| [2023-03-17 10:56:09,735][24380] Avg episode reward: [(0, '22.248')] | |
| [2023-03-17 10:56:09,863][01277] Saving new best policy, reward=22.610! | |
| [2023-03-17 10:56:10,354][01299] Updated weights for policy 0, policy_version 745 (0.0014) | |
| [2023-03-17 10:56:11,597][01299] Updated weights for policy 0, policy_version 755 (0.0009) | |
| [2023-03-17 10:56:12,894][01299] Updated weights for policy 0, policy_version 765 (0.0010) | |
| [2023-03-17 10:56:14,193][01299] Updated weights for policy 0, policy_version 775 (0.0011) | |
| [2023-03-17 10:56:14,734][24380] Fps is (10 sec: 31948.8, 60 sec: 32153.6, 300 sec: 31647.1). Total num frames: 3190784. Throughput: 0: 8051.2. Samples: 739416. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0) | |
| [2023-03-17 10:56:14,735][24380] Avg episode reward: [(0, '22.320')] | |
| [2023-03-17 10:56:15,457][01299] Updated weights for policy 0, policy_version 785 (0.0011) | |
| [2023-03-17 10:56:16,726][01299] Updated weights for policy 0, policy_version 795 (0.0014) | |
| [2023-03-17 10:56:17,985][01299] Updated weights for policy 0, policy_version 805 (0.0010) | |
| [2023-03-17 10:56:19,278][01299] Updated weights for policy 0, policy_version 815 (0.0010) | |
| [2023-03-17 10:56:19,734][24380] Fps is (10 sec: 31948.6, 60 sec: 32153.6, 300 sec: 31662.1). Total num frames: 3350528. Throughput: 0: 8047.0. Samples: 787348. Policy #0 lag: (min: 0.0, avg: 1.4, max: 3.0) | |
| [2023-03-17 10:56:19,735][24380] Avg episode reward: [(0, '24.349')] | |
| [2023-03-17 10:56:19,863][01277] Saving new best policy, reward=24.642! | |
| [2023-03-17 10:56:20,556][01299] Updated weights for policy 0, policy_version 825 (0.0010) | |
| [2023-03-17 10:56:21,799][01299] Updated weights for policy 0, policy_version 835 (0.0012) | |
| [2023-03-17 10:56:23,077][01299] Updated weights for policy 0, policy_version 845 (0.0015) | |
| [2023-03-17 10:56:24,359][01299] Updated weights for policy 0, policy_version 855 (0.0012) | |
| [2023-03-17 10:56:24,734][24380] Fps is (10 sec: 32358.4, 60 sec: 32222.2, 300 sec: 31714.8). Total num frames: 3514368. Throughput: 0: 8045.7. Samples: 811526. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) | |
| [2023-03-17 10:56:24,735][24380] Avg episode reward: [(0, '22.221')] | |
| [2023-03-17 10:56:25,633][01299] Updated weights for policy 0, policy_version 865 (0.0010) | |
| [2023-03-17 10:56:26,920][01299] Updated weights for policy 0, policy_version 875 (0.0011) | |
| [2023-03-17 10:56:28,186][01299] Updated weights for policy 0, policy_version 885 (0.0012) | |
| [2023-03-17 10:56:29,474][01299] Updated weights for policy 0, policy_version 895 (0.0010) | |
| [2023-03-17 10:56:29,739][24380] Fps is (10 sec: 32342.2, 60 sec: 32219.2, 300 sec: 31724.0). Total num frames: 3674112. Throughput: 0: 8048.2. Samples: 859876. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) | |
| [2023-03-17 10:56:29,741][24380] Avg episode reward: [(0, '24.978')] | |
| [2023-03-17 10:56:29,746][01277] Saving /home/ckahmann/RL/train_dir/default_experiment/checkpoint_p0/checkpoint_000000897_3674112.pth... | |
| [2023-03-17 10:56:29,863][01277] Saving new best policy, reward=26.247! | |
| [2023-03-17 10:56:30,766][01299] Updated weights for policy 0, policy_version 905 (0.0009) | |
| [2023-03-17 10:56:32,032][01299] Updated weights for policy 0, policy_version 915 (0.0010) | |
| [2023-03-17 10:56:33,354][01299] Updated weights for policy 0, policy_version 925 (0.0010) | |
| [2023-03-17 10:56:34,610][01299] Updated weights for policy 0, policy_version 935 (0.0012) | |
| [2023-03-17 10:56:34,734][24380] Fps is (10 sec: 31948.6, 60 sec: 32221.9, 300 sec: 31735.1). Total num frames: 3833856. Throughput: 0: 8028.9. Samples: 907774. Policy #0 lag: (min: 0.0, avg: 1.2, max: 2.0) | |
| [2023-03-17 10:56:34,735][24380] Avg episode reward: [(0, '24.749')] | |
| [2023-03-17 10:56:35,882][01299] Updated weights for policy 0, policy_version 945 (0.0013) | |
| [2023-03-17 10:56:37,173][01299] Updated weights for policy 0, policy_version 955 (0.0011) | |
| [2023-03-17 10:56:38,451][01299] Updated weights for policy 0, policy_version 965 (0.0011) | |
| [2023-03-17 10:56:39,707][01299] Updated weights for policy 0, policy_version 975 (0.0012) | |
| [2023-03-17 10:56:39,734][24380] Fps is (10 sec: 31964.8, 60 sec: 32153.6, 300 sec: 31744.0). Total num frames: 3993600. Throughput: 0: 8021.1. Samples: 931680. Policy #0 lag: (min: 0.0, avg: 1.4, max: 3.0) | |
| [2023-03-17 10:56:39,735][24380] Avg episode reward: [(0, '24.199')] | |
| [2023-03-17 10:56:40,085][01277] Stopping Batcher_0... | |
| [2023-03-17 10:56:40,086][01277] Loop batcher_evt_loop terminating... | |
| [2023-03-17 10:56:40,087][01277] Saving /home/ckahmann/RL/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2023-03-17 10:56:40,085][24380] Component Batcher_0 stopped! | |
| [2023-03-17 10:56:40,096][01334] Stopping RolloutWorker_w14... | |
| [2023-03-17 10:56:40,097][01328] Stopping RolloutWorker_w10... | |
| [2023-03-17 10:56:40,097][01334] Loop rollout_proc14_evt_loop terminating... | |
| [2023-03-17 10:56:40,097][01328] Loop rollout_proc10_evt_loop terminating... | |
| [2023-03-17 10:56:40,097][01319] Stopping RolloutWorker_w2... | |
| [2023-03-17 10:56:40,097][24380] Component RolloutWorker_w14 stopped! | |
| [2023-03-17 10:56:40,098][01301] Stopping RolloutWorker_w1... | |
| [2023-03-17 10:56:40,098][01323] Stopping RolloutWorker_w5... | |
| [2023-03-17 10:56:40,099][01326] Stopping RolloutWorker_w7... | |
| [2023-03-17 10:56:40,098][01324] Stopping RolloutWorker_w6... | |
| [2023-03-17 10:56:40,099][01319] Loop rollout_proc2_evt_loop terminating... | |
| [2023-03-17 10:56:40,099][01301] Loop rollout_proc1_evt_loop terminating... | |
| [2023-03-17 10:56:40,099][01323] Loop rollout_proc5_evt_loop terminating... | |
| [2023-03-17 10:56:40,099][01332] Stopping RolloutWorker_w15... | |
| [2023-03-17 10:56:40,099][01326] Loop rollout_proc7_evt_loop terminating... | |
| [2023-03-17 10:56:40,099][01324] Loop rollout_proc6_evt_loop terminating... | |
| [2023-03-17 10:56:40,099][01332] Loop rollout_proc15_evt_loop terminating... | |
| [2023-03-17 10:56:40,099][01335] Stopping RolloutWorker_w13... | |
| [2023-03-17 10:56:40,100][01335] Loop rollout_proc13_evt_loop terminating... | |
| [2023-03-17 10:56:40,100][01325] Stopping RolloutWorker_w8... | |
| [2023-03-17 10:56:40,099][24380] Component RolloutWorker_w10 stopped! | |
| [2023-03-17 10:56:40,100][01321] Stopping RolloutWorker_w4... | |
| [2023-03-17 10:56:40,101][01320] Stopping RolloutWorker_w3... | |
| [2023-03-17 10:56:40,101][01325] Loop rollout_proc8_evt_loop terminating... | |
| [2023-03-17 10:56:40,101][01320] Loop rollout_proc3_evt_loop terminating... | |
| [2023-03-17 10:56:40,101][01321] Loop rollout_proc4_evt_loop terminating... | |
| [2023-03-17 10:56:40,101][01329] Stopping RolloutWorker_w11... | |
| [2023-03-17 10:56:40,102][24380] Component RolloutWorker_w2 stopped! | |
| [2023-03-17 10:56:40,103][24380] Component RolloutWorker_w1 stopped! | |
| [2023-03-17 10:56:40,104][24380] Component RolloutWorker_w6 stopped! | |
| [2023-03-17 10:56:40,105][24380] Component RolloutWorker_w5 stopped! | |
| [2023-03-17 10:56:40,106][24380] Component RolloutWorker_w7 stopped! | |
| [2023-03-17 10:56:40,107][24380] Component RolloutWorker_w15 stopped! | |
| [2023-03-17 10:56:40,108][24380] Component RolloutWorker_w13 stopped! | |
| [2023-03-17 10:56:40,109][01331] Stopping RolloutWorker_w12... | |
| [2023-03-17 10:56:40,109][24380] Component RolloutWorker_w8 stopped! | |
| [2023-03-17 10:56:40,109][01331] Loop rollout_proc12_evt_loop terminating... | |
| [2023-03-17 10:56:40,110][24380] Component RolloutWorker_w4 stopped! | |
| [2023-03-17 10:56:40,110][24380] Component RolloutWorker_w3 stopped! | |
| [2023-03-17 10:56:40,111][24380] Component RolloutWorker_w11 stopped! | |
| [2023-03-17 10:56:40,112][24380] Component RolloutWorker_w12 stopped! | |
| [2023-03-17 10:56:40,102][01329] Loop rollout_proc11_evt_loop terminating... | |
| [2023-03-17 10:56:40,116][01327] Stopping RolloutWorker_w9... | |
| [2023-03-17 10:56:40,116][01327] Loop rollout_proc9_evt_loop terminating... | |
| [2023-03-17 10:56:40,116][24380] Component RolloutWorker_w9 stopped! | |
| [2023-03-17 10:56:40,117][01300] Stopping RolloutWorker_w0... | |
| [2023-03-17 10:56:40,118][01300] Loop rollout_proc0_evt_loop terminating... | |
| [2023-03-17 10:56:40,117][24380] Component RolloutWorker_w0 stopped! | |
| [2023-03-17 10:56:40,126][01299] Weights refcount: 2 0 | |
| [2023-03-17 10:56:40,127][01299] Stopping InferenceWorker_p0-w0... | |
| [2023-03-17 10:56:40,128][01299] Loop inference_proc0-0_evt_loop terminating... | |
| [2023-03-17 10:56:40,128][24380] Component InferenceWorker_p0-w0 stopped! | |
| [2023-03-17 10:56:40,158][01277] Removing /home/ckahmann/RL/train_dir/default_experiment/checkpoint_p0/checkpoint_000000045_184320.pth | |
| [2023-03-17 10:56:40,165][01277] Saving /home/ckahmann/RL/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2023-03-17 10:56:40,243][01277] Stopping LearnerWorker_p0... | |
| [2023-03-17 10:56:40,243][01277] Loop learner_proc0_evt_loop terminating... | |
| [2023-03-17 10:56:40,243][24380] Component LearnerWorker_p0 stopped! | |
| [2023-03-17 10:56:40,245][24380] Waiting for process learner_proc0 to stop... | |
| [2023-03-17 10:56:40,965][24380] Waiting for process inference_proc0-0 to join... | |
| [2023-03-17 10:56:40,967][24380] Waiting for process rollout_proc0 to join... | |
| [2023-03-17 10:56:40,968][24380] Waiting for process rollout_proc1 to join... | |
| [2023-03-17 10:56:40,970][24380] Waiting for process rollout_proc2 to join... | |
| [2023-03-17 10:56:40,971][24380] Waiting for process rollout_proc3 to join... | |
| [2023-03-17 10:56:40,972][24380] Waiting for process rollout_proc4 to join... | |
| [2023-03-17 10:56:40,974][24380] Waiting for process rollout_proc5 to join... | |
| [2023-03-17 10:56:40,975][24380] Waiting for process rollout_proc6 to join... | |
| [2023-03-17 10:56:40,976][24380] Waiting for process rollout_proc7 to join... | |
| [2023-03-17 10:56:40,978][24380] Waiting for process rollout_proc8 to join... | |
| [2023-03-17 10:56:40,979][24380] Waiting for process rollout_proc9 to join... | |
| [2023-03-17 10:56:40,980][24380] Waiting for process rollout_proc10 to join... | |
| [2023-03-17 10:56:40,982][24380] Waiting for process rollout_proc11 to join... | |
| [2023-03-17 10:56:40,983][24380] Waiting for process rollout_proc12 to join... | |
| [2023-03-17 10:56:40,984][24380] Waiting for process rollout_proc13 to join... | |
| [2023-03-17 10:56:40,986][24380] Waiting for process rollout_proc14 to join... | |
| [2023-03-17 10:56:40,987][24380] Waiting for process rollout_proc15 to join... | |
| [2023-03-17 10:56:40,988][24380] Batcher 0 profile tree view: | |
| batching: 12.2211, releasing_batches: 0.0246 | |
| [2023-03-17 10:56:40,989][24380] InferenceWorker_p0-w0 profile tree view: | |
| wait_policy: 0.0001 | |
| wait_policy_total: 5.2569 | |
| update_model: 2.5986 | |
| weight_update: 0.0012 | |
| one_step: 0.0036 | |
| handle_policy_step: 107.7825 | |
| deserialize: 7.7872, stack: 0.7211, obs_to_device_normalize: 31.9304, forward: 37.9525, send_messages: 9.9044 | |
| prepare_outputs: 14.9006 | |
| to_cpu: 9.4684 | |
| [2023-03-17 10:56:40,990][24380] Learner 0 profile tree view: | |
| misc: 0.0056, prepare_batch: 7.4842 | |
| train: 28.2527 | |
| epoch_init: 0.0042, minibatch_init: 0.0061, losses_postprocess: 0.1527, kl_divergence: 0.1730, after_optimizer: 0.3012 | |
| calculate_losses: 8.7205 | |
| losses_init: 0.0027, forward_head: 0.7555, bptt_initial: 5.6775, tail: 0.4340, advantages_returns: 0.1187, losses: 0.6629 | |
| bptt: 0.9171 | |
| bptt_forward_core: 0.8796 | |
| update: 18.5538 | |
| clip: 0.8257 | |
| [2023-03-17 10:56:40,992][24380] RolloutWorker_w0 profile tree view: | |
| wait_for_trajectories: 0.0611, enqueue_policy_requests: 4.1572, env_step: 51.2831, overhead: 4.9311, complete_rollouts: 0.1200 | |
| save_policy_outputs: 4.2128 | |
| split_output_tensors: 2.0315 | |
| [2023-03-17 10:56:40,993][24380] RolloutWorker_w15 profile tree view: | |
| wait_for_trajectories: 0.0640, enqueue_policy_requests: 4.2558, env_step: 53.0790, overhead: 5.1241, complete_rollouts: 0.1230 | |
| save_policy_outputs: 4.2865 | |
| split_output_tensors: 2.0872 | |
| [2023-03-17 10:56:40,995][24380] Loop Runner_EvtLoop terminating... | |
| [2023-03-17 10:56:40,996][24380] Runner profile tree view: | |
| main_loop: 129.7535 | |
| [2023-03-17 10:56:40,998][24380] Collected {0: 4005888}, FPS: 29452.5 | |
| [2023-03-17 10:58:47,379][24380] Loading existing experiment configuration from /home/ckahmann/RL/train_dir/default_experiment/config.json | |
| [2023-03-17 10:58:47,380][24380] Overriding arg 'num_workers' with value 1 passed from command line | |
| [2023-03-17 10:58:47,381][24380] Adding new argument 'no_render'=True that is not in the saved config file! | |
| [2023-03-17 10:58:47,381][24380] Adding new argument 'save_video'=True that is not in the saved config file! | |
| [2023-03-17 10:58:47,382][24380] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! | |
| [2023-03-17 10:58:47,382][24380] Adding new argument 'video_name'=None that is not in the saved config file! | |
| [2023-03-17 10:58:47,383][24380] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file! | |
| [2023-03-17 10:58:47,384][24380] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! | |
| [2023-03-17 10:58:47,384][24380] Adding new argument 'push_to_hub'=False that is not in the saved config file! | |
| [2023-03-17 10:58:47,385][24380] Adding new argument 'hf_repository'=None that is not in the saved config file! | |
| [2023-03-17 10:58:47,385][24380] Adding new argument 'policy_index'=0 that is not in the saved config file! | |
| [2023-03-17 10:58:47,386][24380] Adding new argument 'eval_deterministic'=False that is not in the saved config file! | |
| [2023-03-17 10:58:47,387][24380] Adding new argument 'train_script'=None that is not in the saved config file! | |
| [2023-03-17 10:58:47,387][24380] Adding new argument 'enjoy_script'=None that is not in the saved config file! | |
| [2023-03-17 10:58:47,388][24380] Using frameskip 1 and render_action_repeat=4 for evaluation | |
| [2023-03-17 10:58:47,404][24380] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-03-17 10:58:47,406][24380] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-03-17 10:58:47,407][24380] RunningMeanStd input shape: (1,) | |
| [2023-03-17 10:58:47,419][24380] ConvEncoder: input_channels=3 | |
| [2023-03-17 10:58:47,538][24380] Conv encoder output size: 512 | |
| [2023-03-17 10:58:47,539][24380] Policy head output size: 512 | |
| [2023-03-17 10:58:49,022][24380] Loading state from checkpoint /home/ckahmann/RL/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2023-03-17 10:58:49,781][24380] Num frames 100... | |
| [2023-03-17 10:58:49,931][24380] Num frames 200... | |
| [2023-03-17 10:58:50,058][24380] Num frames 300... | |
| [2023-03-17 10:58:50,169][24380] Num frames 400... | |
| [2023-03-17 10:58:50,290][24380] Num frames 500... | |
| [2023-03-17 10:58:50,397][24380] Num frames 600... | |
| [2023-03-17 10:58:50,508][24380] Num frames 700... | |
| [2023-03-17 10:58:50,619][24380] Num frames 800... | |
| [2023-03-17 10:58:50,731][24380] Num frames 900... | |
| [2023-03-17 10:58:50,841][24380] Num frames 1000... | |
| [2023-03-17 10:58:50,956][24380] Num frames 1100... | |
| [2023-03-17 10:58:51,077][24380] Num frames 1200... | |
| [2023-03-17 10:58:51,193][24380] Avg episode rewards: #0: 27.480, true rewards: #0: 12.480 | |
| [2023-03-17 10:58:51,195][24380] Avg episode reward: 27.480, avg true_objective: 12.480 | |
| [2023-03-17 10:58:51,292][24380] Num frames 1300... | |
| [2023-03-17 10:58:51,422][24380] Num frames 1400... | |
| [2023-03-17 10:58:51,579][24380] Num frames 1500... | |
| [2023-03-17 10:58:51,702][24380] Num frames 1600... | |
| [2023-03-17 10:58:51,863][24380] Avg episode rewards: #0: 17.405, true rewards: #0: 8.405 | |
| [2023-03-17 10:58:51,865][24380] Avg episode reward: 17.405, avg true_objective: 8.405 | |
| [2023-03-17 10:58:51,906][24380] Num frames 1700... | |
| [2023-03-17 10:58:52,054][24380] Num frames 1800... | |
| [2023-03-17 10:58:52,167][24380] Num frames 1900... | |
| [2023-03-17 10:58:52,273][24380] Avg episode rewards: #0: 12.457, true rewards: #0: 6.457 | |
| [2023-03-17 10:58:52,275][24380] Avg episode reward: 12.457, avg true_objective: 6.457 | |
| [2023-03-17 10:58:52,380][24380] Num frames 2000... | |
| [2023-03-17 10:58:52,503][24380] Num frames 2100... | |
| [2023-03-17 10:58:52,627][24380] Num frames 2200... | |
| [2023-03-17 10:58:52,753][24380] Num frames 2300... | |
| [2023-03-17 10:58:52,874][24380] Num frames 2400... | |
| [2023-03-17 10:58:53,011][24380] Num frames 2500... | |
| [2023-03-17 10:58:53,176][24380] Num frames 2600... | |
| [2023-03-17 10:58:53,288][24380] Num frames 2700... | |
| [2023-03-17 10:58:53,369][24380] Num frames 2800... | |
| [2023-03-17 10:58:53,476][24380] Avg episode rewards: #0: 14.173, true rewards: #0: 7.172 | |
| [2023-03-17 10:58:53,478][24380] Avg episode reward: 14.173, avg true_objective: 7.172 | |
| [2023-03-17 10:58:53,523][24380] Num frames 2900... | |
| [2023-03-17 10:58:53,607][24380] Num frames 3000... | |
| [2023-03-17 10:58:53,690][24380] Num frames 3100... | |
| [2023-03-17 10:58:53,808][24380] Num frames 3200... | |
| [2023-03-17 10:58:53,936][24380] Num frames 3300... | |
| [2023-03-17 10:58:54,055][24380] Num frames 3400... | |
| [2023-03-17 10:58:54,190][24380] Num frames 3500... | |
| [2023-03-17 10:58:54,288][24380] Num frames 3600... | |
| [2023-03-17 10:58:54,368][24380] Num frames 3700... | |
| [2023-03-17 10:58:54,444][24380] Num frames 3800... | |
| [2023-03-17 10:58:54,520][24380] Num frames 3900... | |
| [2023-03-17 10:58:54,596][24380] Num frames 4000... | |
| [2023-03-17 10:58:54,674][24380] Num frames 4100... | |
| [2023-03-17 10:58:54,802][24380] Avg episode rewards: #0: 16.590, true rewards: #0: 8.390 | |
| [2023-03-17 10:58:54,803][24380] Avg episode reward: 16.590, avg true_objective: 8.390 | |
| [2023-03-17 10:58:54,816][24380] Num frames 4200... | |
| [2023-03-17 10:58:54,914][24380] Num frames 4300... | |
| [2023-03-17 10:58:54,995][24380] Num frames 4400... | |
| [2023-03-17 10:58:55,071][24380] Num frames 4500... | |
| [2023-03-17 10:58:55,147][24380] Num frames 4600... | |
| [2023-03-17 10:58:55,224][24380] Num frames 4700... | |
| [2023-03-17 10:58:55,301][24380] Num frames 4800... | |
| [2023-03-17 10:58:55,358][24380] Avg episode rewards: #0: 15.338, true rewards: #0: 8.005 | |
| [2023-03-17 10:58:55,359][24380] Avg episode reward: 15.338, avg true_objective: 8.005 | |
| [2023-03-17 10:58:55,457][24380] Num frames 4900... | |
| [2023-03-17 10:58:55,539][24380] Num frames 5000... | |
| [2023-03-17 10:58:55,621][24380] Num frames 5100... | |
| [2023-03-17 10:58:55,702][24380] Num frames 5200... | |
| [2023-03-17 10:58:55,786][24380] Num frames 5300... | |
| [2023-03-17 10:58:55,868][24380] Num frames 5400... | |
| [2023-03-17 10:58:55,950][24380] Num frames 5500... | |
| [2023-03-17 10:58:56,032][24380] Num frames 5600... | |
| [2023-03-17 10:58:56,115][24380] Num frames 5700... | |
| [2023-03-17 10:58:56,199][24380] Num frames 5800... | |
| [2023-03-17 10:58:56,328][24380] Num frames 5900... | |
| [2023-03-17 10:58:56,458][24380] Num frames 6000... | |
| [2023-03-17 10:58:56,592][24380] Num frames 6100... | |
| [2023-03-17 10:58:56,721][24380] Num frames 6200... | |
| [2023-03-17 10:58:56,860][24380] Num frames 6300... | |
| [2023-03-17 10:58:56,995][24380] Num frames 6400... | |
| [2023-03-17 10:58:57,127][24380] Num frames 6500... | |
| [2023-03-17 10:58:57,249][24380] Num frames 6600... | |
| [2023-03-17 10:58:57,333][24380] Num frames 6700... | |
| [2023-03-17 10:58:57,411][24380] Num frames 6800... | |
| [2023-03-17 10:58:57,489][24380] Num frames 6900... | |
| [2023-03-17 10:58:57,546][24380] Avg episode rewards: #0: 21.861, true rewards: #0: 9.861 | |
| [2023-03-17 10:58:57,547][24380] Avg episode reward: 21.861, avg true_objective: 9.861 | |
| [2023-03-17 10:58:57,644][24380] Num frames 7000... | |
| [2023-03-17 10:58:57,748][24380] Num frames 7100... | |
| [2023-03-17 10:58:57,876][24380] Num frames 7200... | |
| [2023-03-17 10:58:57,992][24380] Num frames 7300... | |
| [2023-03-17 10:58:58,072][24380] Avg episode rewards: #0: 19.899, true rewards: #0: 9.149 | |
| [2023-03-17 10:58:58,074][24380] Avg episode reward: 19.899, avg true_objective: 9.149 | |
| [2023-03-17 10:58:58,196][24380] Num frames 7400... | |
| [2023-03-17 10:58:58,399][24380] Num frames 7500... | |
| [2023-03-17 10:58:58,543][24380] Num frames 7600... | |
| [2023-03-17 10:58:58,673][24380] Num frames 7700... | |
| [2023-03-17 10:58:58,776][24380] Num frames 7800... | |
| [2023-03-17 10:58:58,908][24380] Num frames 7900... | |
| [2023-03-17 10:58:59,066][24380] Num frames 8000... | |
| [2023-03-17 10:58:59,182][24380] Num frames 8100... | |
| [2023-03-17 10:58:59,294][24380] Num frames 8200... | |
| [2023-03-17 10:58:59,427][24380] Num frames 8300... | |
| [2023-03-17 10:58:59,546][24380] Num frames 8400... | |
| [2023-03-17 10:58:59,665][24380] Num frames 8500... | |
| [2023-03-17 10:58:59,797][24380] Num frames 8600... | |
| [2023-03-17 10:58:59,917][24380] Num frames 8700... | |
| [2023-03-17 10:59:00,031][24380] Num frames 8800... | |
| [2023-03-17 10:59:00,161][24380] Num frames 8900... | |
| [2023-03-17 10:59:00,323][24380] Num frames 9000... | |
| [2023-03-17 10:59:00,458][24380] Num frames 9100... | |
| [2023-03-17 10:59:00,596][24380] Num frames 9200... | |
| [2023-03-17 10:59:00,665][24380] Avg episode rewards: #0: 22.674, true rewards: #0: 10.230 | |
| [2023-03-17 10:59:00,667][24380] Avg episode reward: 22.674, avg true_objective: 10.230 | |
| [2023-03-17 10:59:00,847][24380] Num frames 9300... | |
| [2023-03-17 10:59:00,983][24380] Num frames 9400... | |
| [2023-03-17 10:59:01,101][24380] Num frames 9500... | |
| [2023-03-17 10:59:01,238][24380] Num frames 9600... | |
| [2023-03-17 10:59:01,396][24380] Num frames 9700... | |
| [2023-03-17 10:59:01,527][24380] Num frames 9800... | |
| [2023-03-17 10:59:01,665][24380] Num frames 9900... | |
| [2023-03-17 10:59:01,806][24380] Num frames 10000... | |
| [2023-03-17 10:59:01,927][24380] Num frames 10100... | |
| [2023-03-17 10:59:02,057][24380] Num frames 10200... | |
| [2023-03-17 10:59:02,180][24380] Num frames 10300... | |
| [2023-03-17 10:59:02,343][24380] Num frames 10400... | |
| [2023-03-17 10:59:02,541][24380] Num frames 10500... | |
| [2023-03-17 10:59:02,701][24380] Avg episode rewards: #0: 23.768, true rewards: #0: 10.568 | |
| [2023-03-17 10:59:02,703][24380] Avg episode reward: 23.768, avg true_objective: 10.568 | |
| [2023-03-17 10:59:30,255][24380] Replay video saved to /home/ckahmann/RL/train_dir/default_experiment/replay.mp4! | |
| [2023-03-17 11:02:12,760][24380] Loading existing experiment configuration from /home/ckahmann/RL/train_dir/default_experiment/config.json | |
| [2023-03-17 11:02:12,761][24380] Overriding arg 'num_workers' with value 1 passed from command line | |
| [2023-03-17 11:02:12,762][24380] Adding new argument 'no_render'=True that is not in the saved config file! | |
| [2023-03-17 11:02:12,762][24380] Adding new argument 'save_video'=True that is not in the saved config file! | |
| [2023-03-17 11:02:12,763][24380] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! | |
| [2023-03-17 11:02:12,764][24380] Adding new argument 'video_name'=None that is not in the saved config file! | |
| [2023-03-17 11:02:12,764][24380] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! | |
| [2023-03-17 11:02:12,765][24380] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! | |
| [2023-03-17 11:02:12,766][24380] Adding new argument 'push_to_hub'=True that is not in the saved config file! | |
| [2023-03-17 11:02:12,766][24380] Adding new argument 'hf_repository'='Christian90/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! | |
| [2023-03-17 11:02:12,767][24380] Adding new argument 'policy_index'=0 that is not in the saved config file! | |
| [2023-03-17 11:02:12,767][24380] Adding new argument 'eval_deterministic'=False that is not in the saved config file! | |
| [2023-03-17 11:02:12,768][24380] Adding new argument 'train_script'=None that is not in the saved config file! | |
| [2023-03-17 11:02:12,769][24380] Adding new argument 'enjoy_script'=None that is not in the saved config file! | |
| [2023-03-17 11:02:12,769][24380] Using frameskip 1 and render_action_repeat=4 for evaluation | |
| [2023-03-17 11:02:12,788][24380] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-03-17 11:02:12,790][24380] RunningMeanStd input shape: (1,) | |
| [2023-03-17 11:02:12,800][24380] ConvEncoder: input_channels=3 | |
| [2023-03-17 11:02:12,828][24380] Conv encoder output size: 512 | |
| [2023-03-17 11:02:12,829][24380] Policy head output size: 512 | |
| [2023-03-17 11:02:12,863][24380] Loading state from checkpoint /home/ckahmann/RL/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2023-03-17 11:02:13,294][24380] Num frames 100... | |
| [2023-03-17 11:02:13,426][24380] Num frames 200... | |
| [2023-03-17 11:02:13,543][24380] Num frames 300... | |
| [2023-03-17 11:02:13,650][24380] Num frames 400... | |
| [2023-03-17 11:02:13,770][24380] Num frames 500... | |
| [2023-03-17 11:02:13,905][24380] Num frames 600... | |
| [2023-03-17 11:02:14,026][24380] Num frames 700... | |
| [2023-03-17 11:02:14,133][24380] Num frames 800... | |
| [2023-03-17 11:02:14,262][24380] Avg episode rewards: #0: 20.640, true rewards: #0: 8.640 | |
| [2023-03-17 11:02:14,263][24380] Avg episode reward: 20.640, avg true_objective: 8.640 | |
| [2023-03-17 11:02:14,334][24380] Num frames 900... | |
| [2023-03-17 11:02:14,461][24380] Num frames 1000... | |
| [2023-03-17 11:02:14,598][24380] Num frames 1100... | |
| [2023-03-17 11:02:14,723][24380] Num frames 1200... | |
| [2023-03-17 11:02:14,888][24380] Num frames 1300... | |
| [2023-03-17 11:02:15,068][24380] Num frames 1400... | |
| [2023-03-17 11:02:15,252][24380] Num frames 1500... | |
| [2023-03-17 11:02:15,427][24380] Num frames 1600... | |
| [2023-03-17 11:02:15,554][24380] Num frames 1700... | |
| [2023-03-17 11:02:15,688][24380] Num frames 1800... | |
| [2023-03-17 11:02:15,828][24380] Num frames 1900... | |
| [2023-03-17 11:02:15,969][24380] Num frames 2000... | |
| [2023-03-17 11:02:16,097][24380] Num frames 2100... | |
| [2023-03-17 11:02:16,221][24380] Num frames 2200... | |
| [2023-03-17 11:02:16,361][24380] Num frames 2300... | |
| [2023-03-17 11:02:16,472][24380] Num frames 2400... | |
| [2023-03-17 11:02:16,599][24380] Num frames 2500... | |
| [2023-03-17 11:02:16,701][24380] Avg episode rewards: #0: 31.140, true rewards: #0: 12.640 | |
| [2023-03-17 11:02:16,703][24380] Avg episode reward: 31.140, avg true_objective: 12.640 | |
| [2023-03-17 11:02:16,836][24380] Num frames 2600... | |
| [2023-03-17 11:02:16,977][24380] Num frames 2700... | |
| [2023-03-17 11:02:17,108][24380] Num frames 2800... | |
| [2023-03-17 11:02:17,235][24380] Num frames 2900... | |
| [2023-03-17 11:02:17,353][24380] Num frames 3000... | |
| [2023-03-17 11:02:17,480][24380] Num frames 3100... | |
| [2023-03-17 11:02:17,601][24380] Num frames 3200... | |
| [2023-03-17 11:02:17,731][24380] Num frames 3300... | |
| [2023-03-17 11:02:17,850][24380] Num frames 3400... | |
| [2023-03-17 11:02:17,991][24380] Num frames 3500... | |
| [2023-03-17 11:02:18,135][24380] Num frames 3600... | |
| [2023-03-17 11:02:18,286][24380] Num frames 3700... | |
| [2023-03-17 11:02:18,405][24380] Num frames 3800... | |
| [2023-03-17 11:02:18,555][24380] Num frames 3900... | |
| [2023-03-17 11:02:18,677][24380] Num frames 4000... | |
| [2023-03-17 11:02:18,795][24380] Num frames 4100... | |
| [2023-03-17 11:02:18,932][24380] Num frames 4200... | |
| [2023-03-17 11:02:19,128][24380] Avg episode rewards: #0: 36.960, true rewards: #0: 14.293 | |
| [2023-03-17 11:02:19,130][24380] Avg episode reward: 36.960, avg true_objective: 14.293 | |
| [2023-03-17 11:02:19,160][24380] Num frames 4300... | |
| [2023-03-17 11:02:19,297][24380] Num frames 4400... | |
| [2023-03-17 11:02:19,439][24380] Num frames 4500... | |
| [2023-03-17 11:02:19,577][24380] Num frames 4600... | |
| [2023-03-17 11:02:19,698][24380] Num frames 4700... | |
| [2023-03-17 11:02:19,832][24380] Num frames 4800... | |
| [2023-03-17 11:02:19,884][24380] Avg episode rewards: #0: 29.750, true rewards: #0: 12.000 | |
| [2023-03-17 11:02:19,886][24380] Avg episode reward: 29.750, avg true_objective: 12.000 | |
| [2023-03-17 11:02:20,021][24380] Num frames 4900... | |
| [2023-03-17 11:02:20,140][24380] Num frames 5000... | |
| [2023-03-17 11:02:20,255][24380] Num frames 5100... | |
| [2023-03-17 11:02:20,373][24380] Num frames 5200... | |
| [2023-03-17 11:02:20,492][24380] Num frames 5300... | |
| [2023-03-17 11:02:20,635][24380] Num frames 5400... | |
| [2023-03-17 11:02:20,764][24380] Num frames 5500... | |
| [2023-03-17 11:02:20,892][24380] Num frames 5600... | |
| [2023-03-17 11:02:21,008][24380] Num frames 5700... | |
| [2023-03-17 11:02:21,126][24380] Avg episode rewards: #0: 28.088, true rewards: #0: 11.488 | |
| [2023-03-17 11:02:21,128][24380] Avg episode reward: 28.088, avg true_objective: 11.488 | |
| [2023-03-17 11:02:21,232][24380] Num frames 5800... | |
| [2023-03-17 11:02:21,376][24380] Num frames 5900... | |
| [2023-03-17 11:02:21,540][24380] Num frames 6000... | |
| [2023-03-17 11:02:21,661][24380] Num frames 6100... | |
| [2023-03-17 11:02:21,809][24380] Num frames 6200... | |
| [2023-03-17 11:02:21,940][24380] Num frames 6300... | |
| [2023-03-17 11:02:22,059][24380] Num frames 6400... | |
| [2023-03-17 11:02:22,172][24380] Num frames 6500... | |
| [2023-03-17 11:02:22,285][24380] Num frames 6600... | |
| [2023-03-17 11:02:22,414][24380] Num frames 6700... | |
| [2023-03-17 11:02:22,534][24380] Num frames 6800... | |
| [2023-03-17 11:02:22,647][24380] Num frames 6900... | |
| [2023-03-17 11:02:22,762][24380] Num frames 7000... | |
| [2023-03-17 11:02:22,885][24380] Num frames 7100... | |
| [2023-03-17 11:02:23,036][24380] Num frames 7200... | |
| [2023-03-17 11:02:23,152][24380] Num frames 7300... | |
| [2023-03-17 11:02:23,292][24380] Num frames 7400... | |
| [2023-03-17 11:02:23,418][24380] Num frames 7500... | |
| [2023-03-17 11:02:23,504][24380] Num frames 7600... | |
| [2023-03-17 11:02:23,589][24380] Num frames 7700... | |
| [2023-03-17 11:02:23,717][24380] Num frames 7800... | |
| [2023-03-17 11:02:23,830][24380] Avg episode rewards: #0: 32.406, true rewards: #0: 13.073 | |
| [2023-03-17 11:02:23,832][24380] Avg episode reward: 32.406, avg true_objective: 13.073 | |
| [2023-03-17 11:02:23,947][24380] Num frames 7900... | |
| [2023-03-17 11:02:24,092][24380] Num frames 8000... | |
| [2023-03-17 11:02:24,247][24380] Num frames 8100... | |
| [2023-03-17 11:02:24,367][24380] Num frames 8200... | |
| [2023-03-17 11:02:24,487][24380] Num frames 8300... | |
| [2023-03-17 11:02:24,609][24380] Num frames 8400... | |
| [2023-03-17 11:02:24,724][24380] Num frames 8500... | |
| [2023-03-17 11:02:24,835][24380] Num frames 8600... | |
| [2023-03-17 11:02:24,998][24380] Avg episode rewards: #0: 29.965, true rewards: #0: 12.394 | |
| [2023-03-17 11:02:25,000][24380] Avg episode reward: 29.965, avg true_objective: 12.394 | |
| [2023-03-17 11:02:25,050][24380] Num frames 8700... | |
| [2023-03-17 11:02:25,182][24380] Num frames 8800... | |
| [2023-03-17 11:02:25,289][24380] Num frames 8900... | |
| [2023-03-17 11:02:25,402][24380] Num frames 9000... | |
| [2023-03-17 11:02:25,569][24380] Num frames 9100... | |
| [2023-03-17 11:02:25,681][24380] Num frames 9200... | |
| [2023-03-17 11:02:25,829][24380] Num frames 9300... | |
| [2023-03-17 11:02:26,014][24380] Num frames 9400... | |
| [2023-03-17 11:02:26,142][24380] Num frames 9500... | |
| [2023-03-17 11:02:26,264][24380] Num frames 9600... | |
| [2023-03-17 11:02:26,370][24380] Avg episode rewards: #0: 28.929, true rewards: #0: 12.054 | |
| [2023-03-17 11:02:26,372][24380] Avg episode reward: 28.929, avg true_objective: 12.054 | |
| [2023-03-17 11:02:26,492][24380] Num frames 9700... | |
| [2023-03-17 11:02:26,650][24380] Num frames 9800... | |
| [2023-03-17 11:02:26,798][24380] Num frames 9900... | |
| [2023-03-17 11:02:26,914][24380] Num frames 10000... | |
| [2023-03-17 11:02:27,105][24380] Num frames 10100... | |
| [2023-03-17 11:02:27,295][24380] Num frames 10200... | |
| [2023-03-17 11:02:27,483][24380] Num frames 10300... | |
| [2023-03-17 11:02:27,662][24380] Num frames 10400... | |
| [2023-03-17 11:02:27,784][24380] Num frames 10500... | |
| [2023-03-17 11:02:27,902][24380] Num frames 10600... | |
| [2023-03-17 11:02:27,963][24380] Avg episode rewards: #0: 28.670, true rewards: #0: 11.781 | |
| [2023-03-17 11:02:27,965][24380] Avg episode reward: 28.670, avg true_objective: 11.781 | |
| [2023-03-17 11:02:28,112][24380] Num frames 10700... | |
| [2023-03-17 11:02:28,238][24380] Num frames 10800... | |
| [2023-03-17 11:02:28,364][24380] Num frames 10900... | |
| [2023-03-17 11:02:28,495][24380] Num frames 11000... | |
| [2023-03-17 11:02:28,621][24380] Num frames 11100... | |
| [2023-03-17 11:02:28,756][24380] Num frames 11200... | |
| [2023-03-17 11:02:28,887][24380] Num frames 11300... | |
| [2023-03-17 11:02:29,022][24380] Num frames 11400... | |
| [2023-03-17 11:02:29,149][24380] Num frames 11500... | |
| [2023-03-17 11:02:29,269][24380] Num frames 11600... | |
| [2023-03-17 11:02:29,429][24380] Avg episode rewards: #0: 28.480, true rewards: #0: 11.680 | |
| [2023-03-17 11:02:29,431][24380] Avg episode reward: 28.480, avg true_objective: 11.680 | |
| [2023-03-17 11:02:59,990][24380] Replay video saved to /home/ckahmann/RL/train_dir/default_experiment/replay.mp4! | |