| [2023-02-24 07:55:59,139][784615] Saving configuration to /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/config.json... | |
| [2023-02-24 07:55:59,305][784615] Rollout worker 0 uses device cpu | |
| [2023-02-24 07:55:59,305][784615] Rollout worker 1 uses device cpu | |
| [2023-02-24 07:55:59,306][784615] Rollout worker 2 uses device cpu | |
| [2023-02-24 07:55:59,306][784615] Rollout worker 3 uses device cpu | |
| [2023-02-24 07:55:59,306][784615] Rollout worker 4 uses device cpu | |
| [2023-02-24 07:55:59,307][784615] Rollout worker 5 uses device cpu | |
| [2023-02-24 07:55:59,307][784615] Rollout worker 6 uses device cpu | |
| [2023-02-24 07:55:59,308][784615] Rollout worker 7 uses device cpu | |
| [2023-02-24 07:55:59,357][784615] Using GPUs [0] for process 0 (actually maps to GPUs [1]) | |
| [2023-02-24 07:55:59,358][784615] InferenceWorker_p0-w0: min num requests: 2 | |
| [2023-02-24 07:55:59,378][784615] Starting all processes... | |
| [2023-02-24 07:55:59,378][784615] Starting process learner_proc0 | |
| [2023-02-24 07:55:59,428][784615] Starting all processes... | |
| [2023-02-24 07:55:59,438][784615] Starting process inference_proc0-0 | |
| [2023-02-24 07:55:59,439][784615] Starting process rollout_proc0 | |
| [2023-02-24 07:55:59,439][784615] Starting process rollout_proc1 | |
| [2023-02-24 07:55:59,439][784615] Starting process rollout_proc2 | |
| [2023-02-24 07:55:59,439][784615] Starting process rollout_proc3 | |
| [2023-02-24 07:55:59,439][784615] Starting process rollout_proc4 | |
| [2023-02-24 07:55:59,439][784615] Starting process rollout_proc5 | |
| [2023-02-24 07:55:59,440][784615] Starting process rollout_proc6 | |
| [2023-02-24 07:55:59,441][784615] Starting process rollout_proc7 | |
| [2023-02-24 07:56:00,842][794035] Worker 3 uses CPU cores [3] | |
| [2023-02-24 07:56:00,862][794036] Worker 5 uses CPU cores [5] | |
| [2023-02-24 07:56:00,978][794037] Worker 4 uses CPU cores [4] | |
| [2023-02-24 07:56:00,998][794038] Worker 2 uses CPU cores [2] | |
| [2023-02-24 07:56:00,999][794019] Low niceness requires sudo! | |
| [2023-02-24 07:56:00,999][794019] Using GPUs [0] for process 0 (actually maps to GPUs [1]) | |
| [2023-02-24 07:56:01,000][794019] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [0]) for learning process 0 | |
| [2023-02-24 07:56:01,022][794032] Low niceness requires sudo! | |
| [2023-02-24 07:56:01,022][794032] Using GPUs [0] for process 0 (actually maps to GPUs [1]) | |
| [2023-02-24 07:56:01,022][794032] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [0]) for inference process 0 | |
| [2023-02-24 07:56:01,025][794019] Num visible devices: 1 | |
| [2023-02-24 07:56:01,040][794032] Num visible devices: 1 | |
| [2023-02-24 07:56:01,050][794019] Starting seed is not provided | |
| [2023-02-24 07:56:01,050][794019] Using GPUs [0] for process 0 (actually maps to GPUs [1]) | |
| [2023-02-24 07:56:01,050][794019] Initializing actor-critic model on device cuda:0 | |
| [2023-02-24 07:56:01,051][794019] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-02-24 07:56:01,051][794019] RunningMeanStd input shape: (1,) | |
| [2023-02-24 07:56:01,058][794040] Worker 7 uses CPU cores [7] | |
| [2023-02-24 07:56:01,065][794019] ConvEncoder: input_channels=3 | |
| [2023-02-24 07:56:01,154][794034] Worker 1 uses CPU cores [1] | |
| [2023-02-24 07:56:01,186][794039] Worker 6 uses CPU cores [6] | |
| [2023-02-24 07:56:01,194][794033] Worker 0 uses CPU cores [0] | |
| [2023-02-24 07:56:01,198][794019] Conv encoder output size: 512 | |
| [2023-02-24 07:56:01,198][794019] Policy head output size: 512 | |
| [2023-02-24 07:56:01,216][794019] Created Actor Critic model with architecture: | |
| [2023-02-24 07:56:01,216][794019] ActorCriticSharedWeights( | |
| (obs_normalizer): ObservationNormalizer( | |
| (running_mean_std): RunningMeanStdDictInPlace( | |
| (running_mean_std): ModuleDict( | |
| (obs): RunningMeanStdInPlace() | |
| ) | |
| ) | |
| ) | |
| (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) | |
| (encoder): VizdoomEncoder( | |
| (basic_encoder): ConvEncoder( | |
| (enc): RecursiveScriptModule( | |
| original_name=ConvEncoderImpl | |
| (conv_head): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Conv2d) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| (2): RecursiveScriptModule(original_name=Conv2d) | |
| (3): RecursiveScriptModule(original_name=ELU) | |
| (4): RecursiveScriptModule(original_name=Conv2d) | |
| (5): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| (mlp_layers): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Linear) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| ) | |
| ) | |
| ) | |
| (core): ModelCoreRNN( | |
| (core): GRU(512, 512) | |
| ) | |
| (decoder): MlpDecoder( | |
| (mlp): Identity() | |
| ) | |
| (critic_linear): Linear(in_features=512, out_features=1, bias=True) | |
| (action_parameterization): ActionParameterizationDefault( | |
| (distribution_linear): Linear(in_features=512, out_features=5, bias=True) | |
| ) | |
| ) | |
| [2023-02-24 07:56:04,122][794019] Using optimizer <class 'torch.optim.adam.Adam'> | |
| [2023-02-24 07:56:04,122][794019] No checkpoints found | |
| [2023-02-24 07:56:04,122][794019] Did not load from checkpoint, starting from scratch! | |
| [2023-02-24 07:56:04,122][794019] Initialized policy 0 weights for model version 0 | |
| [2023-02-24 07:56:04,124][794019] LearnerWorker_p0 finished initialization! | |
| [2023-02-24 07:56:04,124][794019] Using GPUs [0] for process 0 (actually maps to GPUs [1]) | |
| [2023-02-24 07:56:05,229][794032] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-02-24 07:56:05,229][794032] RunningMeanStd input shape: (1,) | |
| [2023-02-24 07:56:05,237][794032] ConvEncoder: input_channels=3 | |
| [2023-02-24 07:56:05,307][794032] Conv encoder output size: 512 | |
| [2023-02-24 07:56:05,307][794032] Policy head output size: 512 | |
| [2023-02-24 07:56:06,350][784615] Inference worker 0-0 is ready! | |
| [2023-02-24 07:56:06,350][784615] All inference workers are ready! Signal rollout workers to start! | |
| [2023-02-24 07:56:06,367][794036] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-24 07:56:06,368][794039] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-24 07:56:06,368][794034] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-24 07:56:06,368][794035] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-24 07:56:06,369][794038] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-24 07:56:06,374][794040] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-24 07:56:06,390][794037] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-24 07:56:06,393][794033] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-24 07:56:06,669][794035] Decorrelating experience for 0 frames... | |
| [2023-02-24 07:56:06,672][794039] Decorrelating experience for 0 frames... | |
| [2023-02-24 07:56:06,678][794036] Decorrelating experience for 0 frames... | |
| [2023-02-24 07:56:06,681][794037] Decorrelating experience for 0 frames... | |
| [2023-02-24 07:56:06,683][794040] Decorrelating experience for 0 frames... | |
| [2023-02-24 07:56:06,929][794035] Decorrelating experience for 32 frames... | |
| [2023-02-24 07:56:06,946][794040] Decorrelating experience for 32 frames... | |
| [2023-02-24 07:56:06,958][794036] Decorrelating experience for 32 frames... | |
| [2023-02-24 07:56:06,977][794037] Decorrelating experience for 32 frames... | |
| [2023-02-24 07:56:06,986][794038] Decorrelating experience for 0 frames... | |
| [2023-02-24 07:56:06,997][794039] Decorrelating experience for 32 frames... | |
| [2023-02-24 07:56:07,033][794033] Decorrelating experience for 0 frames... | |
| [2023-02-24 07:56:07,227][794038] Decorrelating experience for 32 frames... | |
| [2023-02-24 07:56:07,268][794034] Decorrelating experience for 0 frames... | |
| [2023-02-24 07:56:07,269][794036] Decorrelating experience for 64 frames... | |
| [2023-02-24 07:56:07,278][794037] Decorrelating experience for 64 frames... | |
| [2023-02-24 07:56:07,293][794033] Decorrelating experience for 32 frames... | |
| [2023-02-24 07:56:07,502][794038] Decorrelating experience for 64 frames... | |
| [2023-02-24 07:56:07,545][794039] Decorrelating experience for 64 frames... | |
| [2023-02-24 07:56:07,584][794036] Decorrelating experience for 96 frames... | |
| [2023-02-24 07:56:07,591][794033] Decorrelating experience for 64 frames... | |
| [2023-02-24 07:56:07,594][794040] Decorrelating experience for 64 frames... | |
| [2023-02-24 07:56:07,600][794037] Decorrelating experience for 96 frames... | |
| [2023-02-24 07:56:07,628][794034] Decorrelating experience for 32 frames... | |
| [2023-02-24 07:56:07,822][794033] Decorrelating experience for 96 frames... | |
| [2023-02-24 07:56:07,846][794039] Decorrelating experience for 96 frames... | |
| [2023-02-24 07:56:07,890][794038] Decorrelating experience for 96 frames... | |
| [2023-02-24 07:56:07,895][794040] Decorrelating experience for 96 frames... | |
| [2023-02-24 07:56:08,065][794035] Decorrelating experience for 64 frames... | |
| [2023-02-24 07:56:08,153][784615] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2023-02-24 07:56:08,319][794034] Decorrelating experience for 64 frames... | |
| [2023-02-24 07:56:08,572][794035] Decorrelating experience for 96 frames... | |
| [2023-02-24 07:56:08,577][794034] Decorrelating experience for 96 frames... | |
| [2023-02-24 07:56:10,130][794019] Signal inference workers to stop experience collection... | |
| [2023-02-24 07:56:10,137][794032] InferenceWorker_p0-w0: stopping experience collection | |
| [2023-02-24 07:56:12,636][794019] Signal inference workers to resume experience collection... | |
| [2023-02-24 07:56:12,637][794032] InferenceWorker_p0-w0: resuming experience collection | |
| [2023-02-24 07:56:13,153][784615] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 819.2). Total num frames: 4096. Throughput: 0: 494.8. Samples: 2474. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) | |
| [2023-02-24 07:56:13,154][784615] Avg episode reward: [(0, '2.494')] | |
| [2023-02-24 07:56:15,144][794032] Updated weights for policy 0, policy_version 10 (0.0249) | |
| [2023-02-24 07:56:17,664][794032] Updated weights for policy 0, policy_version 20 (0.0006) | |
| [2023-02-24 07:56:18,153][784615] Fps is (10 sec: 8601.6, 60 sec: 8601.6, 300 sec: 8601.6). Total num frames: 86016. Throughput: 0: 1166.6. Samples: 11666. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-24 07:56:18,154][784615] Avg episode reward: [(0, '4.437')] | |
| [2023-02-24 07:56:19,351][784615] Heartbeat connected on Batcher_0 | |
| [2023-02-24 07:56:19,354][784615] Heartbeat connected on LearnerWorker_p0 | |
| [2023-02-24 07:56:19,362][784615] Heartbeat connected on InferenceWorker_p0-w0 | |
| [2023-02-24 07:56:19,364][784615] Heartbeat connected on RolloutWorker_w0 | |
| [2023-02-24 07:56:19,365][784615] Heartbeat connected on RolloutWorker_w1 | |
| [2023-02-24 07:56:19,368][784615] Heartbeat connected on RolloutWorker_w3 | |
| [2023-02-24 07:56:19,373][784615] Heartbeat connected on RolloutWorker_w4 | |
| [2023-02-24 07:56:19,374][784615] Heartbeat connected on RolloutWorker_w5 | |
| [2023-02-24 07:56:19,375][784615] Heartbeat connected on RolloutWorker_w2 | |
| [2023-02-24 07:56:19,377][784615] Heartbeat connected on RolloutWorker_w7 | |
| [2023-02-24 07:56:19,378][784615] Heartbeat connected on RolloutWorker_w6 | |
| [2023-02-24 07:56:20,182][794032] Updated weights for policy 0, policy_version 30 (0.0006) | |
| [2023-02-24 07:56:22,465][794032] Updated weights for policy 0, policy_version 40 (0.0007) | |
| [2023-02-24 07:56:23,153][784615] Fps is (10 sec: 16793.7, 60 sec: 11468.8, 300 sec: 11468.8). Total num frames: 172032. Throughput: 0: 2462.0. Samples: 36930. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-02-24 07:56:23,154][784615] Avg episode reward: [(0, '4.501')] | |
| [2023-02-24 07:56:23,155][794019] Saving new best policy, reward=4.501! | |
| [2023-02-24 07:56:25,053][794032] Updated weights for policy 0, policy_version 50 (0.0007) | |
| [2023-02-24 07:56:27,490][794032] Updated weights for policy 0, policy_version 60 (0.0007) | |
| [2023-02-24 07:56:28,153][784615] Fps is (10 sec: 16793.6, 60 sec: 12697.6, 300 sec: 12697.6). Total num frames: 253952. Throughput: 0: 3086.0. Samples: 61720. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-02-24 07:56:28,154][784615] Avg episode reward: [(0, '4.402')] | |
| [2023-02-24 07:56:30,062][794032] Updated weights for policy 0, policy_version 70 (0.0007) | |
| [2023-02-24 07:56:32,640][794032] Updated weights for policy 0, policy_version 80 (0.0007) | |
| [2023-02-24 07:56:33,153][784615] Fps is (10 sec: 15974.4, 60 sec: 13271.1, 300 sec: 13271.1). Total num frames: 331776. Throughput: 0: 2951.7. Samples: 73792. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-02-24 07:56:33,154][784615] Avg episode reward: [(0, '4.289')] | |
| [2023-02-24 07:56:35,246][794032] Updated weights for policy 0, policy_version 90 (0.0007) | |
| [2023-02-24 07:56:37,855][794032] Updated weights for policy 0, policy_version 100 (0.0008) | |
| [2023-02-24 07:56:38,153][784615] Fps is (10 sec: 15974.5, 60 sec: 13789.9, 300 sec: 13789.9). Total num frames: 413696. Throughput: 0: 3249.9. Samples: 97496. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-02-24 07:56:38,154][784615] Avg episode reward: [(0, '4.457')] | |
| [2023-02-24 07:56:40,454][794032] Updated weights for policy 0, policy_version 110 (0.0007) | |
| [2023-02-24 07:56:43,061][794032] Updated weights for policy 0, policy_version 120 (0.0006) | |
| [2023-02-24 07:56:43,153][784615] Fps is (10 sec: 15973.9, 60 sec: 14043.3, 300 sec: 14043.3). Total num frames: 491520. Throughput: 0: 3461.9. Samples: 121166. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-02-24 07:56:43,154][784615] Avg episode reward: [(0, '4.401')] | |
| [2023-02-24 07:56:45,699][794032] Updated weights for policy 0, policy_version 130 (0.0007) | |
| [2023-02-24 07:56:48,153][784615] Fps is (10 sec: 15564.7, 60 sec: 14233.6, 300 sec: 14233.6). Total num frames: 569344. Throughput: 0: 3324.0. Samples: 132960. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-02-24 07:56:48,154][784615] Avg episode reward: [(0, '4.667')] | |
| [2023-02-24 07:56:48,157][794019] Saving new best policy, reward=4.667! | |
| [2023-02-24 07:56:48,334][794032] Updated weights for policy 0, policy_version 140 (0.0007) | |
| [2023-02-24 07:56:50,891][794032] Updated weights for policy 0, policy_version 150 (0.0006) | |
| [2023-02-24 07:56:53,153][784615] Fps is (10 sec: 15565.3, 60 sec: 14381.5, 300 sec: 14381.5). Total num frames: 647168. Throughput: 0: 3474.6. Samples: 156358. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-02-24 07:56:53,154][784615] Avg episode reward: [(0, '4.622')] | |
| [2023-02-24 07:56:53,523][794032] Updated weights for policy 0, policy_version 160 (0.0008) | |
| [2023-02-24 07:56:56,158][794032] Updated weights for policy 0, policy_version 170 (0.0007) | |
| [2023-02-24 07:56:58,153][784615] Fps is (10 sec: 15565.0, 60 sec: 14499.9, 300 sec: 14499.9). Total num frames: 724992. Throughput: 0: 3940.0. Samples: 179774. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-02-24 07:56:58,153][784615] Avg episode reward: [(0, '4.603')] | |
| [2023-02-24 07:56:58,738][794032] Updated weights for policy 0, policy_version 180 (0.0008) | |
| [2023-02-24 07:57:01,298][794032] Updated weights for policy 0, policy_version 190 (0.0007) | |
| [2023-02-24 07:57:03,153][784615] Fps is (10 sec: 15974.3, 60 sec: 14671.1, 300 sec: 14671.1). Total num frames: 806912. Throughput: 0: 3998.0. Samples: 191576. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-02-24 07:57:03,154][784615] Avg episode reward: [(0, '4.850')] | |
| [2023-02-24 07:57:03,155][794019] Saving new best policy, reward=4.850! | |
| [2023-02-24 07:57:03,931][794032] Updated weights for policy 0, policy_version 200 (0.0007) | |
| [2023-02-24 07:57:06,517][794032] Updated weights for policy 0, policy_version 210 (0.0007) | |
| [2023-02-24 07:57:08,153][784615] Fps is (10 sec: 15974.2, 60 sec: 14745.6, 300 sec: 14745.6). Total num frames: 884736. Throughput: 0: 3958.6. Samples: 215068. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-24 07:57:08,154][784615] Avg episode reward: [(0, '5.203')] | |
| [2023-02-24 07:57:08,156][794019] Saving new best policy, reward=5.203! | |
| [2023-02-24 07:57:09,207][794032] Updated weights for policy 0, policy_version 220 (0.0006) | |
| [2023-02-24 07:57:11,788][794032] Updated weights for policy 0, policy_version 230 (0.0006) | |
| [2023-02-24 07:57:13,153][784615] Fps is (10 sec: 15564.9, 60 sec: 15974.4, 300 sec: 14808.6). Total num frames: 962560. Throughput: 0: 3932.8. Samples: 238698. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-02-24 07:57:13,154][784615] Avg episode reward: [(0, '4.831')] | |
| [2023-02-24 07:57:14,400][794032] Updated weights for policy 0, policy_version 240 (0.0006) | |
| [2023-02-24 07:57:17,007][794032] Updated weights for policy 0, policy_version 250 (0.0008) | |
| [2023-02-24 07:57:18,153][784615] Fps is (10 sec: 15564.8, 60 sec: 15906.1, 300 sec: 14862.6). Total num frames: 1040384. Throughput: 0: 3928.0. Samples: 250550. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-02-24 07:57:18,154][784615] Avg episode reward: [(0, '6.010')] | |
| [2023-02-24 07:57:18,157][794019] Saving new best policy, reward=6.010! | |
| [2023-02-24 07:57:19,591][794032] Updated weights for policy 0, policy_version 260 (0.0006) | |
| [2023-02-24 07:57:21,628][784615] Keyboard interrupt detected in the event loop EvtLoop [Runner_EvtLoop, process=main process 784615], exiting... | |
| [2023-02-24 07:57:21,629][794019] Stopping Batcher_0... | |
| [2023-02-24 07:57:21,629][794019] Loop batcher_evt_loop terminating... | |
| [2023-02-24 07:57:21,628][784615] Runner profile tree view: | |
| main_loop: 82.2510 | |
| [2023-02-24 07:57:21,630][784615] Collected {0: 1093632}, FPS: 13296.3 | |
| [2023-02-24 07:57:21,638][794039] Stopping RolloutWorker_w6... | |
| [2023-02-24 07:57:21,638][794035] Stopping RolloutWorker_w3... | |
| [2023-02-24 07:57:21,638][794035] Loop rollout_proc3_evt_loop terminating... | |
| [2023-02-24 07:57:21,638][794034] Stopping RolloutWorker_w1... | |
| [2023-02-24 07:57:21,639][794039] Loop rollout_proc6_evt_loop terminating... | |
| [2023-02-24 07:57:21,639][794034] Loop rollout_proc1_evt_loop terminating... | |
| [2023-02-24 07:57:21,645][794033] Stopping RolloutWorker_w0... | |
| [2023-02-24 07:57:21,646][794033] Loop rollout_proc0_evt_loop terminating... | |
| [2023-02-24 07:57:21,646][794037] Stopping RolloutWorker_w4... | |
| [2023-02-24 07:57:21,647][794037] Loop rollout_proc4_evt_loop terminating... | |
| [2023-02-24 07:57:21,647][794038] Stopping RolloutWorker_w2... | |
| [2023-02-24 07:57:21,648][794038] Loop rollout_proc2_evt_loop terminating... | |
| [2023-02-24 07:57:21,651][794036] Stopping RolloutWorker_w5... | |
| [2023-02-24 07:57:21,651][794036] Loop rollout_proc5_evt_loop terminating... | |
| [2023-02-24 07:57:21,652][794040] Stopping RolloutWorker_w7... | |
| [2023-02-24 07:57:21,653][794040] Loop rollout_proc7_evt_loop terminating... | |
| [2023-02-24 07:57:21,667][794019] Saving /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000000268_1097728.pth... | |
| [2023-02-24 07:57:21,688][794032] Weights refcount: 2 0 | |
| [2023-02-24 07:57:21,695][794032] Stopping InferenceWorker_p0-w0... | |
| [2023-02-24 07:57:21,696][794032] Loop inference_proc0-0_evt_loop terminating... | |
| [2023-02-24 07:57:21,825][794019] Stopping LearnerWorker_p0... | |
| [2023-02-24 07:57:21,826][794019] Loop learner_proc0_evt_loop terminating... | |
| [2023-02-24 07:57:37,644][784615] Loading existing experiment configuration from /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/config.json | |
| [2023-02-24 07:57:37,644][784615] Overriding arg 'num_workers' with value 1 passed from command line | |
| [2023-02-24 07:57:37,645][784615] Adding new argument 'no_render'=True that is not in the saved config file! | |
| [2023-02-24 07:57:37,645][784615] Adding new argument 'save_video'=True that is not in the saved config file! | |
| [2023-02-24 07:57:37,645][784615] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! | |
| [2023-02-24 07:57:37,646][784615] Adding new argument 'video_name'=None that is not in the saved config file! | |
| [2023-02-24 07:57:37,646][784615] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! | |
| [2023-02-24 07:57:37,646][784615] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! | |
| [2023-02-24 07:57:37,647][784615] Adding new argument 'push_to_hub'=True that is not in the saved config file! | |
| [2023-02-24 07:57:37,647][784615] Adding new argument 'hf_repository'='chqmatteo/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! | |
| [2023-02-24 07:57:37,647][784615] Adding new argument 'policy_index'=0 that is not in the saved config file! | |
| [2023-02-24 07:57:37,648][784615] Adding new argument 'eval_deterministic'=False that is not in the saved config file! | |
| [2023-02-24 07:57:37,648][784615] Adding new argument 'train_script'=None that is not in the saved config file! | |
| [2023-02-24 07:57:37,648][784615] Adding new argument 'enjoy_script'=None that is not in the saved config file! | |
| [2023-02-24 07:57:37,649][784615] Using frameskip 1 and render_action_repeat=4 for evaluation | |
| [2023-02-24 07:57:37,655][784615] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-24 07:57:37,656][784615] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-02-24 07:57:37,657][784615] RunningMeanStd input shape: (1,) | |
| [2023-02-24 07:57:37,665][784615] ConvEncoder: input_channels=3 | |
| [2023-02-24 07:57:37,755][784615] Conv encoder output size: 512 | |
| [2023-02-24 07:57:37,755][784615] Policy head output size: 512 | |
| [2023-02-24 07:57:40,369][784615] Loading state from checkpoint /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000000268_1097728.pth... | |
| [2023-02-24 07:57:42,531][784615] Num frames 100... | |
| [2023-02-24 07:57:42,595][784615] Num frames 200... | |
| [2023-02-24 07:57:42,663][784615] Num frames 300... | |
| [2023-02-24 07:57:42,732][784615] Num frames 400... | |
| [2023-02-24 07:57:42,832][784615] Avg episode rewards: #0: 6.800, true rewards: #0: 4.800 | |
| [2023-02-24 07:57:42,833][784615] Avg episode reward: 6.800, avg true_objective: 4.800 | |
| [2023-02-24 07:57:42,848][784615] Num frames 500... | |
| [2023-02-24 07:57:42,911][784615] Num frames 600... | |
| [2023-02-24 07:57:42,969][784615] Num frames 700... | |
| [2023-02-24 07:57:43,043][784615] Num frames 800... | |
| [2023-02-24 07:57:43,113][784615] Avg episode rewards: #0: 5.630, true rewards: #0: 4.130 | |
| [2023-02-24 07:57:43,113][784615] Avg episode reward: 5.630, avg true_objective: 4.130 | |
| [2023-02-24 07:57:43,175][784615] Num frames 900... | |
| [2023-02-24 07:57:43,244][784615] Num frames 1000... | |
| [2023-02-24 07:57:43,309][784615] Num frames 1100... | |
| [2023-02-24 07:57:43,371][784615] Num frames 1200... | |
| [2023-02-24 07:57:43,470][784615] Avg episode rewards: #0: 5.580, true rewards: #0: 4.247 | |
| [2023-02-24 07:57:43,471][784615] Avg episode reward: 5.580, avg true_objective: 4.247 | |
| [2023-02-24 07:57:43,487][784615] Num frames 1300... | |
| [2023-02-24 07:57:43,564][784615] Num frames 1400... | |
| [2023-02-24 07:57:43,628][784615] Num frames 1500... | |
| [2023-02-24 07:57:43,689][784615] Num frames 1600... | |
| [2023-02-24 07:57:43,781][784615] Avg episode rewards: #0: 5.145, true rewards: #0: 4.145 | |
| [2023-02-24 07:57:43,781][784615] Avg episode reward: 5.145, avg true_objective: 4.145 | |
| [2023-02-24 07:57:43,817][784615] Num frames 1700... | |
| [2023-02-24 07:57:43,887][784615] Num frames 1800... | |
| [2023-02-24 07:57:43,956][784615] Num frames 1900... | |
| [2023-02-24 07:57:44,024][784615] Num frames 2000... | |
| [2023-02-24 07:57:44,089][784615] Num frames 2100... | |
| [2023-02-24 07:57:44,147][784615] Avg episode rewards: #0: 5.612, true rewards: #0: 4.212 | |
| [2023-02-24 07:57:44,148][784615] Avg episode reward: 5.612, avg true_objective: 4.212 | |
| [2023-02-24 07:57:44,208][784615] Num frames 2200... | |
| [2023-02-24 07:57:44,288][784615] Num frames 2300... | |
| [2023-02-24 07:57:44,350][784615] Num frames 2400... | |
| [2023-02-24 07:57:44,463][784615] Avg episode rewards: #0: 5.317, true rewards: #0: 4.150 | |
| [2023-02-24 07:57:44,463][784615] Avg episode reward: 5.317, avg true_objective: 4.150 | |
| [2023-02-24 07:57:44,471][784615] Num frames 2500... | |
| [2023-02-24 07:57:44,540][784615] Num frames 2600... | |
| [2023-02-24 07:57:44,607][784615] Num frames 2700... | |
| [2023-02-24 07:57:44,671][784615] Num frames 2800... | |
| [2023-02-24 07:57:44,738][784615] Num frames 2900... | |
| [2023-02-24 07:57:44,810][784615] Num frames 3000... | |
| [2023-02-24 07:57:44,865][784615] Avg episode rewards: #0: 5.717, true rewards: #0: 4.289 | |
| [2023-02-24 07:57:44,866][784615] Avg episode reward: 5.717, avg true_objective: 4.289 | |
| [2023-02-24 07:57:44,925][784615] Num frames 3100... | |
| [2023-02-24 07:57:44,988][784615] Num frames 3200... | |
| [2023-02-24 07:57:45,057][784615] Num frames 3300... | |
| [2023-02-24 07:57:45,126][784615] Num frames 3400... | |
| [2023-02-24 07:57:45,200][784615] Num frames 3500... | |
| [2023-02-24 07:57:45,287][784615] Avg episode rewards: #0: 5.933, true rewards: #0: 4.432 | |
| [2023-02-24 07:57:45,287][784615] Avg episode reward: 5.933, avg true_objective: 4.432 | |
| [2023-02-24 07:57:45,332][784615] Num frames 3600... | |
| [2023-02-24 07:57:45,410][784615] Num frames 3700... | |
| [2023-02-24 07:57:45,468][784615] Num frames 3800... | |
| [2023-02-24 07:57:45,570][784615] Num frames 3900... | |
| [2023-02-24 07:57:45,634][784615] Num frames 4000... | |
| [2023-02-24 07:57:45,698][784615] Num frames 4100... | |
| [2023-02-24 07:57:45,762][784615] Num frames 4200... | |
| [2023-02-24 07:57:45,873][784615] Avg episode rewards: #0: 6.869, true rewards: #0: 4.758 | |
| [2023-02-24 07:57:45,874][784615] Avg episode reward: 6.869, avg true_objective: 4.758 | |
| [2023-02-24 07:57:45,886][784615] Num frames 4300... | |
| [2023-02-24 07:57:45,952][784615] Num frames 4400... | |
| [2023-02-24 07:57:46,014][784615] Num frames 4500... | |
| [2023-02-24 07:57:46,087][784615] Num frames 4600... | |
| [2023-02-24 07:57:46,149][784615] Num frames 4700... | |
| [2023-02-24 07:57:46,224][784615] Avg episode rewards: #0: 6.730, true rewards: #0: 4.730 | |
| [2023-02-24 07:57:46,225][784615] Avg episode reward: 6.730, avg true_objective: 4.730 | |
| [2023-02-24 07:57:48,353][784615] Replay video saved to /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/replay.mp4! | |
| [2023-02-24 07:58:39,896][784615] Loading existing experiment configuration from /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/config.json | |
| [2023-02-24 07:58:39,896][784615] Overriding arg 'num_workers' with value 1 passed from command line | |
| [2023-02-24 07:58:39,897][784615] Adding new argument 'no_render'=True that is not in the saved config file! | |
| [2023-02-24 07:58:39,897][784615] Adding new argument 'save_video'=True that is not in the saved config file! | |
| [2023-02-24 07:58:39,898][784615] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! | |
| [2023-02-24 07:58:39,898][784615] Adding new argument 'video_name'=None that is not in the saved config file! | |
| [2023-02-24 07:58:39,899][784615] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! | |
| [2023-02-24 07:58:39,899][784615] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! | |
| [2023-02-24 07:58:39,900][784615] Adding new argument 'push_to_hub'=True that is not in the saved config file! | |
| [2023-02-24 07:58:39,900][784615] Adding new argument 'hf_repository'='chqmatteo/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! | |
| [2023-02-24 07:58:39,900][784615] Adding new argument 'policy_index'=0 that is not in the saved config file! | |
| [2023-02-24 07:58:39,901][784615] Adding new argument 'eval_deterministic'=False that is not in the saved config file! | |
| [2023-02-24 07:58:39,901][784615] Adding new argument 'train_script'=None that is not in the saved config file! | |
| [2023-02-24 07:58:39,902][784615] Adding new argument 'enjoy_script'=None that is not in the saved config file! | |
| [2023-02-24 07:58:39,902][784615] Using frameskip 1 and render_action_repeat=4 for evaluation | |
| [2023-02-24 07:58:39,911][784615] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-02-24 07:58:39,912][784615] RunningMeanStd input shape: (1,) | |
| [2023-02-24 07:58:39,919][784615] ConvEncoder: input_channels=3 | |
| [2023-02-24 07:58:39,943][784615] Conv encoder output size: 512 | |
| [2023-02-24 07:58:39,944][784615] Policy head output size: 512 | |
| [2023-02-24 07:58:39,980][784615] Loading state from checkpoint /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000000268_1097728.pth... | |
| [2023-02-24 07:58:40,400][784615] Num frames 100... | |
| [2023-02-24 07:58:40,470][784615] Num frames 200... | |
| [2023-02-24 07:58:40,530][784615] Num frames 300... | |
| [2023-02-24 07:58:40,596][784615] Num frames 400... | |
| [2023-02-24 07:58:40,684][784615] Avg episode rewards: #0: 5.480, true rewards: #0: 4.480 | |
| [2023-02-24 07:58:40,685][784615] Avg episode reward: 5.480, avg true_objective: 4.480 | |
| [2023-02-24 07:58:40,719][784615] Num frames 500... | |
| [2023-02-24 07:58:40,787][784615] Num frames 600... | |
| [2023-02-24 07:58:40,850][784615] Num frames 700... | |
| [2023-02-24 07:58:40,918][784615] Num frames 800... | |
| [2023-02-24 07:58:41,005][784615] Num frames 900... | |
| [2023-02-24 07:58:41,078][784615] Avg episode rewards: #0: 6.640, true rewards: #0: 4.640 | |
| [2023-02-24 07:58:41,079][784615] Avg episode reward: 6.640, avg true_objective: 4.640 | |
| [2023-02-24 07:58:41,129][784615] Num frames 1000... | |
| [2023-02-24 07:58:41,202][784615] Num frames 1100... | |
| [2023-02-24 07:58:41,276][784615] Num frames 1200... | |
| [2023-02-24 07:58:41,360][784615] Num frames 1300... | |
| [2023-02-24 07:58:41,429][784615] Num frames 1400... | |
| [2023-02-24 07:58:41,493][784615] Num frames 1500... | |
| [2023-02-24 07:58:41,554][784615] Num frames 1600... | |
| [2023-02-24 07:58:41,667][784615] Avg episode rewards: #0: 8.653, true rewards: #0: 5.653 | |
| [2023-02-24 07:58:41,668][784615] Avg episode reward: 8.653, avg true_objective: 5.653 | |
| [2023-02-24 07:58:41,673][784615] Num frames 1700... | |
| [2023-02-24 07:58:41,734][784615] Num frames 1800... | |
| [2023-02-24 07:58:41,791][784615] Num frames 1900... | |
| [2023-02-24 07:58:41,848][784615] Num frames 2000... | |
| [2023-02-24 07:58:41,905][784615] Num frames 2100... | |
| [2023-02-24 07:58:41,961][784615] Num frames 2200... | |
| [2023-02-24 07:58:42,055][784615] Avg episode rewards: #0: 8.680, true rewards: #0: 5.680 | |
| [2023-02-24 07:58:42,056][784615] Avg episode reward: 8.680, avg true_objective: 5.680 | |
| [2023-02-24 07:58:42,077][784615] Num frames 2300... | |
| [2023-02-24 07:58:42,141][784615] Num frames 2400... | |
| [2023-02-24 07:58:42,203][784615] Num frames 2500... | |
| [2023-02-24 07:58:42,260][784615] Num frames 2600... | |
| [2023-02-24 07:58:42,346][784615] Avg episode rewards: #0: 7.712, true rewards: #0: 5.312 | |
| [2023-02-24 07:58:42,348][784615] Avg episode reward: 7.712, avg true_objective: 5.312 | |
| [2023-02-24 07:58:42,385][784615] Num frames 2700... | |
| [2023-02-24 07:58:42,449][784615] Num frames 2800... | |
| [2023-02-24 07:58:42,506][784615] Num frames 2900... | |
| [2023-02-24 07:58:42,563][784615] Num frames 3000... | |
| [2023-02-24 07:58:42,621][784615] Num frames 3100... | |
| [2023-02-24 07:58:42,688][784615] Num frames 3200... | |
| [2023-02-24 07:58:42,788][784615] Avg episode rewards: #0: 7.940, true rewards: #0: 5.440 | |
| [2023-02-24 07:58:42,788][784615] Avg episode reward: 7.940, avg true_objective: 5.440 | |
| [2023-02-24 07:58:42,817][784615] Num frames 3300... | |
| [2023-02-24 07:58:42,890][784615] Num frames 3400... | |
| [2023-02-24 07:58:42,964][784615] Num frames 3500... | |
| [2023-02-24 07:58:43,040][784615] Num frames 3600... | |
| [2023-02-24 07:58:43,128][784615] Avg episode rewards: #0: 7.354, true rewards: #0: 5.211 | |
| [2023-02-24 07:58:43,130][784615] Avg episode reward: 7.354, avg true_objective: 5.211 | |
| [2023-02-24 07:58:43,174][784615] Num frames 3700... | |
| [2023-02-24 07:58:43,248][784615] Num frames 3800... | |
| [2023-02-24 07:58:43,316][784615] Num frames 3900... | |
| [2023-02-24 07:58:43,383][784615] Num frames 4000... | |
| [2023-02-24 07:58:43,475][784615] Avg episode rewards: #0: 7.205, true rewards: #0: 5.080 | |
| [2023-02-24 07:58:43,476][784615] Avg episode reward: 7.205, avg true_objective: 5.080 | |
| [2023-02-24 07:58:43,498][784615] Num frames 4100... | |
| [2023-02-24 07:58:43,556][784615] Num frames 4200... | |
| [2023-02-24 07:58:43,614][784615] Num frames 4300... | |
| [2023-02-24 07:58:43,670][784615] Num frames 4400... | |
| [2023-02-24 07:58:43,727][784615] Num frames 4500... | |
| [2023-02-24 07:58:43,790][784615] Num frames 4600... | |
| [2023-02-24 07:58:43,891][784615] Avg episode rewards: #0: 7.413, true rewards: #0: 5.191 | |
| [2023-02-24 07:58:43,891][784615] Avg episode reward: 7.413, avg true_objective: 5.191 | |
| [2023-02-24 07:58:43,912][784615] Num frames 4700... | |
| [2023-02-24 07:58:43,982][784615] Num frames 4800... | |
| [2023-02-24 07:58:44,049][784615] Num frames 4900... | |
| [2023-02-24 07:58:44,117][784615] Num frames 5000... | |
| [2023-02-24 07:58:44,184][784615] Num frames 5100... | |
| [2023-02-24 07:58:44,250][784615] Avg episode rewards: #0: 7.220, true rewards: #0: 5.120 | |
| [2023-02-24 07:58:44,250][784615] Avg episode reward: 7.220, avg true_objective: 5.120 | |
| [2023-02-24 07:58:46,584][784615] Replay video saved to /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/replay.mp4! | |
| [2023-02-24 07:59:12,402][784615] The model has been pushed to https://huggingface.co/chqmatteo/rl_course_vizdoom_health_gathering_supreme | |
| [2023-02-24 08:02:31,136][784615] Environment doom_basic already registered, overwriting... | |
| [2023-02-24 08:02:31,137][784615] Environment doom_two_colors_easy already registered, overwriting... | |
| [2023-02-24 08:02:31,137][784615] Environment doom_two_colors_hard already registered, overwriting... | |
| [2023-02-24 08:02:31,138][784615] Environment doom_dm already registered, overwriting... | |
| [2023-02-24 08:02:31,138][784615] Environment doom_dwango5 already registered, overwriting... | |
| [2023-02-24 08:02:31,139][784615] Environment doom_my_way_home_flat_actions already registered, overwriting... | |
| [2023-02-24 08:02:31,139][784615] Environment doom_defend_the_center_flat_actions already registered, overwriting... | |
| [2023-02-24 08:02:31,139][784615] Environment doom_my_way_home already registered, overwriting... | |
| [2023-02-24 08:02:31,140][784615] Environment doom_deadly_corridor already registered, overwriting... | |
| [2023-02-24 08:02:31,140][784615] Environment doom_defend_the_center already registered, overwriting... | |
| [2023-02-24 08:02:31,140][784615] Environment doom_defend_the_line already registered, overwriting... | |
| [2023-02-24 08:02:31,141][784615] Environment doom_health_gathering already registered, overwriting... | |
| [2023-02-24 08:02:31,141][784615] Environment doom_health_gathering_supreme already registered, overwriting... | |
| [2023-02-24 08:02:31,142][784615] Environment doom_battle already registered, overwriting... | |
| [2023-02-24 08:02:31,142][784615] Environment doom_battle2 already registered, overwriting... | |
| [2023-02-24 08:02:31,142][784615] Environment doom_duel_bots already registered, overwriting... | |
| [2023-02-24 08:02:31,142][784615] Environment doom_deathmatch_bots already registered, overwriting... | |
| [2023-02-24 08:02:31,143][784615] Environment doom_duel already registered, overwriting... | |
| [2023-02-24 08:02:31,143][784615] Environment doom_deathmatch_full already registered, overwriting... | |
| [2023-02-24 08:02:31,143][784615] Environment doom_benchmark already registered, overwriting... | |
| [2023-02-24 08:02:31,144][784615] register_encoder_factory: <function make_vizdoom_encoder at 0x7f9fbe7ae680> | |
| [2023-02-24 08:02:31,153][784615] Loading existing experiment configuration from /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/config.json | |
| [2023-02-24 08:02:31,154][784615] Overriding arg 'train_for_env_steps' with value 40000000 passed from command line | |
| [2023-02-24 08:02:31,157][784615] Experiment dir /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment already exists! | |
| [2023-02-24 08:02:31,158][784615] Resuming existing experiment from /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment... | |
| [2023-02-24 08:02:31,158][784615] Weights and Biases integration disabled | |
| [2023-02-24 08:02:31,159][784615] Environment var CUDA_VISIBLE_DEVICES is 1 | |
| [2023-02-24 08:03:09,472][784615] Environment doom_basic already registered, overwriting... | |
| [2023-02-24 08:03:09,474][784615] Environment doom_two_colors_easy already registered, overwriting... | |
| [2023-02-24 08:03:09,475][784615] Environment doom_two_colors_hard already registered, overwriting... | |
| [2023-02-24 08:03:09,476][784615] Environment doom_dm already registered, overwriting... | |
| [2023-02-24 08:03:09,477][784615] Environment doom_dwango5 already registered, overwriting... | |
| [2023-02-24 08:03:09,477][784615] Environment doom_my_way_home_flat_actions already registered, overwriting... | |
| [2023-02-24 08:03:09,478][784615] Environment doom_defend_the_center_flat_actions already registered, overwriting... | |
| [2023-02-24 08:03:09,478][784615] Environment doom_my_way_home already registered, overwriting... | |
| [2023-02-24 08:03:09,479][784615] Environment doom_deadly_corridor already registered, overwriting... | |
| [2023-02-24 08:03:09,479][784615] Environment doom_defend_the_center already registered, overwriting... | |
| [2023-02-24 08:03:09,480][784615] Environment doom_defend_the_line already registered, overwriting... | |
| [2023-02-24 08:03:09,480][784615] Environment doom_health_gathering already registered, overwriting... | |
| [2023-02-24 08:03:09,481][784615] Environment doom_health_gathering_supreme already registered, overwriting... | |
| [2023-02-24 08:03:09,481][784615] Environment doom_battle already registered, overwriting... | |
| [2023-02-24 08:03:09,482][784615] Environment doom_battle2 already registered, overwriting... | |
| [2023-02-24 08:03:09,482][784615] Environment doom_duel_bots already registered, overwriting... | |
| [2023-02-24 08:03:09,483][784615] Environment doom_deathmatch_bots already registered, overwriting... | |
| [2023-02-24 08:03:09,483][784615] Environment doom_duel already registered, overwriting... | |
| [2023-02-24 08:03:09,483][784615] Environment doom_deathmatch_full already registered, overwriting... | |
| [2023-02-24 08:03:09,484][784615] Environment doom_benchmark already registered, overwriting... | |
| [2023-02-24 08:03:09,484][784615] register_encoder_factory: <function make_vizdoom_encoder at 0x7f9fbe7ae680> | |
| [2023-02-24 08:03:09,494][784615] Loading existing experiment configuration from /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/config.json | |
| [2023-02-24 08:03:09,495][784615] Overriding arg 'train_for_env_steps' with value 40000000 passed from command line | |
| [2023-02-24 08:03:09,498][784615] Experiment dir /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment already exists! | |
| [2023-02-24 08:03:09,499][784615] Resuming existing experiment from /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment... | |
| [2023-02-24 08:03:09,499][784615] Weights and Biases integration disabled | |
| [2023-02-24 08:03:09,500][784615] Environment var CUDA_VISIBLE_DEVICES is 1 | |
| [2023-02-24 08:06:20,559][795538] Saving configuration to /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/config.json... | |
| [2023-02-24 08:06:20,725][795538] Rollout worker 0 uses device cpu | |
| [2023-02-24 08:06:20,726][795538] Rollout worker 1 uses device cpu | |
| [2023-02-24 08:06:20,726][795538] Rollout worker 2 uses device cpu | |
| [2023-02-24 08:06:20,727][795538] Rollout worker 3 uses device cpu | |
| [2023-02-24 08:06:20,727][795538] Rollout worker 4 uses device cpu | |
| [2023-02-24 08:06:20,727][795538] Rollout worker 5 uses device cpu | |
| [2023-02-24 08:06:20,728][795538] Rollout worker 6 uses device cpu | |
| [2023-02-24 08:06:20,728][795538] Rollout worker 7 uses device cpu | |
| [2023-02-24 08:06:20,767][795538] Using GPUs [0] for process 0 (actually maps to GPUs [1]) | |
| [2023-02-24 08:06:20,768][795538] InferenceWorker_p0-w0: min num requests: 2 | |
| [2023-02-24 08:06:20,819][795538] Starting all processes... | |
| [2023-02-24 08:06:20,820][795538] Starting process learner_proc0 | |
| [2023-02-24 08:06:20,869][795538] Starting all processes... | |
| [2023-02-24 08:06:20,872][795538] Starting process inference_proc0-0 | |
| [2023-02-24 08:06:20,873][795538] Starting process rollout_proc0 | |
| [2023-02-24 08:06:20,873][795538] Starting process rollout_proc1 | |
| [2023-02-24 08:06:20,873][795538] Starting process rollout_proc2 | |
| [2023-02-24 08:06:20,873][795538] Starting process rollout_proc3 | |
| [2023-02-24 08:06:20,874][795538] Starting process rollout_proc4 | |
| [2023-02-24 08:06:20,874][795538] Starting process rollout_proc5 | |
| [2023-02-24 08:06:20,874][795538] Starting process rollout_proc6 | |
| [2023-02-24 08:06:20,875][795538] Starting process rollout_proc7 | |
| [2023-02-24 08:06:22,179][795628] Worker 1 uses CPU cores [1] | |
| [2023-02-24 08:06:22,214][795634] Worker 6 uses CPU cores [6] | |
| [2023-02-24 08:06:22,319][795626] Low niceness requires sudo! | |
| [2023-02-24 08:06:22,319][795626] Using GPUs [0] for process 0 (actually maps to GPUs [1]) | |
| [2023-02-24 08:06:22,319][795626] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [0]) for inference process 0 | |
| [2023-02-24 08:06:22,336][795626] Num visible devices: 1 | |
| [2023-02-24 08:06:22,343][795613] Low niceness requires sudo! | |
| [2023-02-24 08:06:22,343][795613] Using GPUs [0] for process 0 (actually maps to GPUs [1]) | |
| [2023-02-24 08:06:22,344][795613] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [0]) for learning process 0 | |
| [2023-02-24 08:06:22,361][795613] Num visible devices: 1 | |
| [2023-02-24 08:06:22,366][795632] Worker 5 uses CPU cores [5] | |
| [2023-02-24 08:06:22,394][795613] Starting seed is not provided | |
| [2023-02-24 08:06:22,394][795613] Using GPUs [0] for process 0 (actually maps to GPUs [1]) | |
| [2023-02-24 08:06:22,394][795613] Initializing actor-critic model on device cuda:0 | |
| [2023-02-24 08:06:22,395][795613] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-02-24 08:06:22,395][795613] RunningMeanStd input shape: (1,) | |
| [2023-02-24 08:06:22,409][795613] ConvEncoder: input_channels=3 | |
| [2023-02-24 08:06:22,497][795613] Conv encoder output size: 512 | |
| [2023-02-24 08:06:22,497][795613] Policy head output size: 512 | |
| [2023-02-24 08:06:22,506][795613] Created Actor Critic model with architecture: | |
| [2023-02-24 08:06:22,507][795613] ActorCriticSharedWeights( | |
| (obs_normalizer): ObservationNormalizer( | |
| (running_mean_std): RunningMeanStdDictInPlace( | |
| (running_mean_std): ModuleDict( | |
| (obs): RunningMeanStdInPlace() | |
| ) | |
| ) | |
| ) | |
| (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) | |
| (encoder): VizdoomEncoder( | |
| (basic_encoder): ConvEncoder( | |
| (enc): RecursiveScriptModule( | |
| original_name=ConvEncoderImpl | |
| (conv_head): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Conv2d) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| (2): RecursiveScriptModule(original_name=Conv2d) | |
| (3): RecursiveScriptModule(original_name=ELU) | |
| (4): RecursiveScriptModule(original_name=Conv2d) | |
| (5): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| (mlp_layers): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Linear) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| ) | |
| ) | |
| ) | |
| (core): ModelCoreRNN( | |
| (core): GRU(512, 512) | |
| ) | |
| (decoder): MlpDecoder( | |
| (mlp): Identity() | |
| ) | |
| (critic_linear): Linear(in_features=512, out_features=1, bias=True) | |
| (action_parameterization): ActionParameterizationDefault( | |
| (distribution_linear): Linear(in_features=512, out_features=5, bias=True) | |
| ) | |
| ) | |
| [2023-02-24 08:06:22,514][795631] Worker 4 uses CPU cores [4] | |
| [2023-02-24 08:06:22,542][795633] Worker 7 uses CPU cores [7] | |
| [2023-02-24 08:06:22,548][795630] Worker 3 uses CPU cores [3] | |
| [2023-02-24 08:06:22,590][795629] Worker 2 uses CPU cores [2] | |
| [2023-02-24 08:06:22,604][795627] Worker 0 uses CPU cores [0] | |
| [2023-02-24 08:06:25,136][795613] Using optimizer <class 'torch.optim.adam.Adam'> | |
| [2023-02-24 08:06:25,137][795613] Loading state from checkpoint /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000000268_1097728.pth... | |
| [2023-02-24 08:06:25,196][795613] Loading model from checkpoint | |
| [2023-02-24 08:06:25,198][795613] Loaded experiment state at self.train_step=268, self.env_steps=1097728 | |
| [2023-02-24 08:06:25,198][795613] Initialized policy 0 weights for model version 268 | |
| [2023-02-24 08:06:25,199][795613] LearnerWorker_p0 finished initialization! | |
| [2023-02-24 08:06:25,199][795613] Using GPUs [0] for process 0 (actually maps to GPUs [1]) | |
| [2023-02-24 08:06:26,272][795626] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-02-24 08:06:26,272][795626] RunningMeanStd input shape: (1,) | |
| [2023-02-24 08:06:26,280][795626] ConvEncoder: input_channels=3 | |
| [2023-02-24 08:06:26,341][795626] Conv encoder output size: 512 | |
| [2023-02-24 08:06:26,342][795626] Policy head output size: 512 | |
| [2023-02-24 08:06:27,341][795538] Inference worker 0-0 is ready! | |
| [2023-02-24 08:06:27,341][795538] All inference workers are ready! Signal rollout workers to start! | |
| [2023-02-24 08:06:27,358][795628] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-24 08:06:27,358][795631] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-24 08:06:27,358][795627] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-24 08:06:27,359][795629] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-24 08:06:27,364][795634] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-24 08:06:27,381][795632] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-24 08:06:27,380][795630] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-24 08:06:27,394][795633] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-24 08:06:27,569][795627] Decorrelating experience for 0 frames... | |
| [2023-02-24 08:06:27,619][795634] Decorrelating experience for 0 frames... | |
| [2023-02-24 08:06:27,650][795629] Decorrelating experience for 0 frames... | |
| [2023-02-24 08:06:27,665][795630] Decorrelating experience for 0 frames... | |
| [2023-02-24 08:06:27,805][795634] Decorrelating experience for 32 frames... | |
| [2023-02-24 08:06:27,881][795630] Decorrelating experience for 32 frames... | |
| [2023-02-24 08:06:27,882][795627] Decorrelating experience for 32 frames... | |
| [2023-02-24 08:06:27,882][795632] Decorrelating experience for 0 frames... | |
| [2023-02-24 08:06:28,090][795630] Decorrelating experience for 64 frames... | |
| [2023-02-24 08:06:28,092][795632] Decorrelating experience for 32 frames... | |
| [2023-02-24 08:06:28,106][795627] Decorrelating experience for 64 frames... | |
| [2023-02-24 08:06:28,308][795632] Decorrelating experience for 64 frames... | |
| [2023-02-24 08:06:28,376][795628] Decorrelating experience for 0 frames... | |
| [2023-02-24 08:06:28,383][795630] Decorrelating experience for 96 frames... | |
| [2023-02-24 08:06:28,383][795629] Decorrelating experience for 32 frames... | |
| [2023-02-24 08:06:28,395][795627] Decorrelating experience for 96 frames... | |
| [2023-02-24 08:06:28,625][795628] Decorrelating experience for 32 frames... | |
| [2023-02-24 08:06:28,660][795633] Decorrelating experience for 0 frames... | |
| [2023-02-24 08:06:28,720][795632] Decorrelating experience for 96 frames... | |
| [2023-02-24 08:06:28,729][795631] Decorrelating experience for 0 frames... | |
| [2023-02-24 08:06:28,907][795633] Decorrelating experience for 32 frames... | |
| [2023-02-24 08:06:28,955][795628] Decorrelating experience for 64 frames... | |
| [2023-02-24 08:06:28,975][795634] Decorrelating experience for 64 frames... | |
| [2023-02-24 08:06:29,141][795633] Decorrelating experience for 64 frames... | |
| [2023-02-24 08:06:29,169][795631] Decorrelating experience for 32 frames... | |
| [2023-02-24 08:06:29,237][795628] Decorrelating experience for 96 frames... | |
| [2023-02-24 08:06:29,253][795629] Decorrelating experience for 64 frames... | |
| [2023-02-24 08:06:29,425][795631] Decorrelating experience for 64 frames... | |
| [2023-02-24 08:06:29,531][795629] Decorrelating experience for 96 frames... | |
| [2023-02-24 08:06:29,537][795634] Decorrelating experience for 96 frames... | |
| [2023-02-24 08:06:29,541][795633] Decorrelating experience for 96 frames... | |
| [2023-02-24 08:06:29,684][795631] Decorrelating experience for 96 frames... | |
| [2023-02-24 08:06:29,698][795538] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 1097728. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2023-02-24 08:06:30,658][795613] Signal inference workers to stop experience collection... | |
| [2023-02-24 08:06:30,662][795626] InferenceWorker_p0-w0: stopping experience collection | |
| [2023-02-24 08:06:32,942][795613] Signal inference workers to resume experience collection... | |
| [2023-02-24 08:06:32,942][795626] InferenceWorker_p0-w0: resuming experience collection | |
| [2023-02-24 08:06:34,697][795538] Fps is (10 sec: 5734.4, 60 sec: 5734.4, 300 sec: 5734.4). Total num frames: 1126400. Throughput: 0: 628.8. Samples: 3144. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) | |
| [2023-02-24 08:06:34,699][795538] Avg episode reward: [(0, '5.223')] | |
| [2023-02-24 08:06:35,433][795626] Updated weights for policy 0, policy_version 278 (0.0234) | |
| [2023-02-24 08:06:38,065][795626] Updated weights for policy 0, policy_version 288 (0.0006) | |
| [2023-02-24 08:06:39,698][795538] Fps is (10 sec: 10649.4, 60 sec: 10649.4, 300 sec: 10649.4). Total num frames: 1204224. Throughput: 0: 2626.3. Samples: 26264. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-24 08:06:39,699][795538] Avg episode reward: [(0, '6.797')] | |
| [2023-02-24 08:06:39,702][795613] Saving new best policy, reward=6.797! | |
| [2023-02-24 08:06:40,702][795626] Updated weights for policy 0, policy_version 298 (0.0006) | |
| [2023-02-24 08:06:40,762][795538] Heartbeat connected on Batcher_0 | |
| [2023-02-24 08:06:40,764][795538] Heartbeat connected on LearnerWorker_p0 | |
| [2023-02-24 08:06:40,771][795538] Heartbeat connected on InferenceWorker_p0-w0 | |
| [2023-02-24 08:06:40,775][795538] Heartbeat connected on RolloutWorker_w0 | |
| [2023-02-24 08:06:40,777][795538] Heartbeat connected on RolloutWorker_w2 | |
| [2023-02-24 08:06:40,778][795538] Heartbeat connected on RolloutWorker_w1 | |
| [2023-02-24 08:06:40,779][795538] Heartbeat connected on RolloutWorker_w3 | |
| [2023-02-24 08:06:40,783][795538] Heartbeat connected on RolloutWorker_w4 | |
| [2023-02-24 08:06:40,784][795538] Heartbeat connected on RolloutWorker_w5 | |
| [2023-02-24 08:06:40,819][795538] Heartbeat connected on RolloutWorker_w7 | |
| [2023-02-24 08:06:40,825][795538] Heartbeat connected on RolloutWorker_w6 | |
| [2023-02-24 08:06:43,242][795626] Updated weights for policy 0, policy_version 308 (0.0007) | |
| [2023-02-24 08:06:44,698][795538] Fps is (10 sec: 15564.8, 60 sec: 12288.0, 300 sec: 12288.0). Total num frames: 1282048. Throughput: 0: 2536.4. Samples: 38046. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) | |
| [2023-02-24 08:06:44,698][795538] Avg episode reward: [(0, '7.215')] | |
| [2023-02-24 08:06:44,700][795613] Saving new best policy, reward=7.215! | |
| [2023-02-24 08:06:45,889][795626] Updated weights for policy 0, policy_version 318 (0.0007) | |
| [2023-02-24 08:06:48,495][795626] Updated weights for policy 0, policy_version 328 (0.0007) | |
| [2023-02-24 08:06:49,698][795538] Fps is (10 sec: 15565.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 1359872. Throughput: 0: 3079.6. Samples: 61592. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-02-24 08:06:49,699][795538] Avg episode reward: [(0, '8.493')] | |
| [2023-02-24 08:06:49,702][795613] Saving new best policy, reward=8.493! | |
| [2023-02-24 08:06:51,107][795626] Updated weights for policy 0, policy_version 338 (0.0007) | |
| [2023-02-24 08:06:53,724][795626] Updated weights for policy 0, policy_version 348 (0.0007) | |
| [2023-02-24 08:06:54,698][795538] Fps is (10 sec: 15564.7, 60 sec: 13598.7, 300 sec: 13598.7). Total num frames: 1437696. Throughput: 0: 3404.3. Samples: 85108. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-02-24 08:06:54,699][795538] Avg episode reward: [(0, '10.669')] | |
| [2023-02-24 08:06:54,700][795613] Saving new best policy, reward=10.669! | |
| [2023-02-24 08:06:56,371][795626] Updated weights for policy 0, policy_version 358 (0.0007) | |
| [2023-02-24 08:06:58,972][795626] Updated weights for policy 0, policy_version 368 (0.0007) | |
| [2023-02-24 08:06:59,697][795538] Fps is (10 sec: 15564.8, 60 sec: 13926.4, 300 sec: 13926.4). Total num frames: 1515520. Throughput: 0: 3226.5. Samples: 96796. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-02-24 08:06:59,698][795538] Avg episode reward: [(0, '13.259')] | |
| [2023-02-24 08:06:59,701][795613] Saving new best policy, reward=13.259! | |
| [2023-02-24 08:07:01,613][795626] Updated weights for policy 0, policy_version 378 (0.0007) | |
| [2023-02-24 08:07:04,206][795626] Updated weights for policy 0, policy_version 388 (0.0007) | |
| [2023-02-24 08:07:04,698][795538] Fps is (10 sec: 15564.9, 60 sec: 14160.5, 300 sec: 14160.5). Total num frames: 1593344. Throughput: 0: 3437.5. Samples: 120312. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-02-24 08:07:04,698][795538] Avg episode reward: [(0, '13.989')] | |
| [2023-02-24 08:07:04,699][795613] Saving new best policy, reward=13.989! | |
| [2023-02-24 08:07:06,831][795626] Updated weights for policy 0, policy_version 398 (0.0007) | |
| [2023-02-24 08:07:08,206][795538] Keyboard interrupt detected in the event loop EvtLoop [Runner_EvtLoop, process=main process 795538], exiting... | |
| [2023-02-24 08:07:08,207][795538] Runner profile tree view: | |
| main_loop: 47.3880 | |
| [2023-02-24 08:07:08,208][795613] Stopping Batcher_0... | |
| [2023-02-24 08:07:08,208][795613] Loop batcher_evt_loop terminating... | |
| [2023-02-24 08:07:08,208][795538] Collected {0: 1650688}, FPS: 11668.8 | |
| [2023-02-24 08:07:08,209][795613] Saving /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000000403_1650688.pth... | |
| [2023-02-24 08:07:08,216][795632] Stopping RolloutWorker_w5... | |
| [2023-02-24 08:07:08,216][795632] Loop rollout_proc5_evt_loop terminating... | |
| [2023-02-24 08:07:08,218][795630] Stopping RolloutWorker_w3... | |
| [2023-02-24 08:07:08,218][795630] Loop rollout_proc3_evt_loop terminating... | |
| [2023-02-24 08:07:08,218][795628] Stopping RolloutWorker_w1... | |
| [2023-02-24 08:07:08,218][795633] Stopping RolloutWorker_w7... | |
| [2023-02-24 08:07:08,219][795628] Loop rollout_proc1_evt_loop terminating... | |
| [2023-02-24 08:07:08,219][795633] Loop rollout_proc7_evt_loop terminating... | |
| [2023-02-24 08:07:08,221][795631] Stopping RolloutWorker_w4... | |
| [2023-02-24 08:07:08,221][795631] Loop rollout_proc4_evt_loop terminating... | |
| [2023-02-24 08:07:08,228][795629] Stopping RolloutWorker_w2... | |
| [2023-02-24 08:07:08,228][795629] Loop rollout_proc2_evt_loop terminating... | |
| [2023-02-24 08:07:08,230][795627] Stopping RolloutWorker_w0... | |
| [2023-02-24 08:07:08,230][795634] Stopping RolloutWorker_w6... | |
| [2023-02-24 08:07:08,230][795634] Loop rollout_proc6_evt_loop terminating... | |
| [2023-02-24 08:07:08,230][795627] Loop rollout_proc0_evt_loop terminating... | |
| [2023-02-24 08:07:08,252][795626] Weights refcount: 2 0 | |
| [2023-02-24 08:07:08,256][795626] Stopping InferenceWorker_p0-w0... | |
| [2023-02-24 08:07:08,260][795626] Loop inference_proc0-0_evt_loop terminating... | |
| [2023-02-24 08:07:08,376][795613] Stopping LearnerWorker_p0... | |
| [2023-02-24 08:07:08,377][795613] Loop learner_proc0_evt_loop terminating... | |
| [2023-02-24 08:07:38,064][795538] Loading existing experiment configuration from /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/config.json | |
| [2023-02-24 08:07:38,065][795538] Overriding arg 'num_workers' with value 1 passed from command line | |
| [2023-02-24 08:07:38,065][795538] Adding new argument 'no_render'=True that is not in the saved config file! | |
| [2023-02-24 08:07:38,065][795538] Adding new argument 'save_video'=True that is not in the saved config file! | |
| [2023-02-24 08:07:38,066][795538] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! | |
| [2023-02-24 08:07:38,066][795538] Adding new argument 'video_name'=None that is not in the saved config file! | |
| [2023-02-24 08:07:38,066][795538] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! | |
| [2023-02-24 08:07:38,067][795538] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! | |
| [2023-02-24 08:07:38,067][795538] Adding new argument 'push_to_hub'=True that is not in the saved config file! | |
| [2023-02-24 08:07:38,068][795538] Adding new argument 'hf_repository'='chqmatteo/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! | |
| [2023-02-24 08:07:38,068][795538] Adding new argument 'policy_index'=0 that is not in the saved config file! | |
| [2023-02-24 08:07:38,068][795538] Adding new argument 'eval_deterministic'=False that is not in the saved config file! | |
| [2023-02-24 08:07:38,068][795538] Adding new argument 'train_script'=None that is not in the saved config file! | |
| [2023-02-24 08:07:38,069][795538] Adding new argument 'enjoy_script'=None that is not in the saved config file! | |
| [2023-02-24 08:07:38,069][795538] Using frameskip 1 and render_action_repeat=4 for evaluation | |
| [2023-02-24 08:07:38,076][795538] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-24 08:07:38,077][795538] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-02-24 08:07:38,078][795538] RunningMeanStd input shape: (1,) | |
| [2023-02-24 08:07:38,086][795538] ConvEncoder: input_channels=3 | |
| [2023-02-24 08:07:38,169][795538] Conv encoder output size: 512 | |
| [2023-02-24 08:07:38,170][795538] Policy head output size: 512 | |
| [2023-02-24 08:07:40,757][795538] Loading state from checkpoint /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000000403_1650688.pth... | |
| [2023-02-24 08:07:42,930][795538] Num frames 100... | |
| [2023-02-24 08:07:43,007][795538] Num frames 200... | |
| [2023-02-24 08:07:43,075][795538] Num frames 300... | |
| [2023-02-24 08:07:43,138][795538] Num frames 400... | |
| [2023-02-24 08:07:43,207][795538] Num frames 500... | |
| [2023-02-24 08:07:43,275][795538] Num frames 600... | |
| [2023-02-24 08:07:43,349][795538] Num frames 700... | |
| [2023-02-24 08:07:43,409][795538] Avg episode rewards: #0: 16.090, true rewards: #0: 7.090 | |
| [2023-02-24 08:07:43,410][795538] Avg episode reward: 16.090, avg true_objective: 7.090 | |
| [2023-02-24 08:07:43,472][795538] Num frames 800... | |
| [2023-02-24 08:07:43,536][795538] Num frames 900... | |
| [2023-02-24 08:07:43,605][795538] Num frames 1000... | |
| [2023-02-24 08:07:43,689][795538] Num frames 1100... | |
| [2023-02-24 08:07:43,790][795538] Avg episode rewards: #0: 10.785, true rewards: #0: 5.785 | |
| [2023-02-24 08:07:43,791][795538] Avg episode reward: 10.785, avg true_objective: 5.785 | |
| [2023-02-24 08:07:43,819][795538] Num frames 1200... | |
| [2023-02-24 08:07:43,885][795538] Num frames 1300... | |
| [2023-02-24 08:07:43,956][795538] Num frames 1400... | |
| [2023-02-24 08:07:44,024][795538] Num frames 1500... | |
| [2023-02-24 08:07:44,102][795538] Num frames 1600... | |
| [2023-02-24 08:07:44,182][795538] Num frames 1700... | |
| [2023-02-24 08:07:44,280][795538] Avg episode rewards: #0: 10.217, true rewards: #0: 5.883 | |
| [2023-02-24 08:07:44,280][795538] Avg episode reward: 10.217, avg true_objective: 5.883 | |
| [2023-02-24 08:07:44,305][795538] Num frames 1800... | |
| [2023-02-24 08:07:44,373][795538] Num frames 1900... | |
| [2023-02-24 08:07:44,443][795538] Num frames 2000... | |
| [2023-02-24 08:07:44,519][795538] Num frames 2100... | |
| [2023-02-24 08:07:44,607][795538] Num frames 2200... | |
| [2023-02-24 08:07:44,678][795538] Num frames 2300... | |
| [2023-02-24 08:07:44,753][795538] Num frames 2400... | |
| [2023-02-24 08:07:44,838][795538] Num frames 2500... | |
| [2023-02-24 08:07:44,919][795538] Num frames 2600... | |
| [2023-02-24 08:07:45,012][795538] Avg episode rewards: #0: 12.653, true rewards: #0: 6.652 | |
| [2023-02-24 08:07:45,013][795538] Avg episode reward: 12.653, avg true_objective: 6.652 | |
| [2023-02-24 08:07:45,040][795538] Num frames 2700... | |
| [2023-02-24 08:07:45,115][795538] Num frames 2800... | |
| [2023-02-24 08:07:45,187][795538] Num frames 2900... | |
| [2023-02-24 08:07:45,261][795538] Num frames 3000... | |
| [2023-02-24 08:07:45,381][795538] Num frames 3100... | |
| [2023-02-24 08:07:45,448][795538] Num frames 3200... | |
| [2023-02-24 08:07:45,513][795538] Num frames 3300... | |
| [2023-02-24 08:07:45,581][795538] Num frames 3400... | |
| [2023-02-24 08:07:45,652][795538] Num frames 3500... | |
| [2023-02-24 08:07:45,717][795538] Num frames 3600... | |
| [2023-02-24 08:07:45,790][795538] Num frames 3700... | |
| [2023-02-24 08:07:45,860][795538] Num frames 3800... | |
| [2023-02-24 08:07:45,935][795538] Num frames 3900... | |
| [2023-02-24 08:07:46,006][795538] Num frames 4000... | |
| [2023-02-24 08:07:46,078][795538] Num frames 4100... | |
| [2023-02-24 08:07:46,213][795538] Avg episode rewards: #0: 17.194, true rewards: #0: 8.394 | |
| [2023-02-24 08:07:46,214][795538] Avg episode reward: 17.194, avg true_objective: 8.394 | |
| [2023-02-24 08:07:46,216][795538] Num frames 4200... | |
| [2023-02-24 08:07:46,291][795538] Num frames 4300... | |
| [2023-02-24 08:07:46,369][795538] Num frames 4400... | |
| [2023-02-24 08:07:46,449][795538] Num frames 4500... | |
| [2023-02-24 08:07:46,522][795538] Num frames 4600... | |
| [2023-02-24 08:07:46,591][795538] Num frames 4700... | |
| [2023-02-24 08:07:46,662][795538] Num frames 4800... | |
| [2023-02-24 08:07:46,732][795538] Num frames 4900... | |
| [2023-02-24 08:07:46,831][795538] Avg episode rewards: #0: 16.942, true rewards: #0: 8.275 | |
| [2023-02-24 08:07:46,831][795538] Avg episode reward: 16.942, avg true_objective: 8.275 | |
| [2023-02-24 08:07:46,855][795538] Num frames 5000... | |
| [2023-02-24 08:07:46,919][795538] Num frames 5100... | |
| [2023-02-24 08:07:46,986][795538] Num frames 5200... | |
| [2023-02-24 08:07:47,056][795538] Num frames 5300... | |
| [2023-02-24 08:07:47,141][795538] Num frames 5400... | |
| [2023-02-24 08:07:47,210][795538] Num frames 5500... | |
| [2023-02-24 08:07:47,278][795538] Num frames 5600... | |
| [2023-02-24 08:07:47,382][795538] Avg episode rewards: #0: 16.394, true rewards: #0: 8.109 | |
| [2023-02-24 08:07:47,383][795538] Avg episode reward: 16.394, avg true_objective: 8.109 | |
| [2023-02-24 08:07:47,406][795538] Num frames 5700... | |
| [2023-02-24 08:07:47,483][795538] Num frames 5800... | |
| [2023-02-24 08:07:47,567][795538] Num frames 5900... | |
| [2023-02-24 08:07:47,636][795538] Num frames 6000... | |
| [2023-02-24 08:07:47,706][795538] Num frames 6100... | |
| [2023-02-24 08:07:47,783][795538] Num frames 6200... | |
| [2023-02-24 08:07:47,861][795538] Num frames 6300... | |
| [2023-02-24 08:07:47,935][795538] Num frames 6400... | |
| [2023-02-24 08:07:48,009][795538] Num frames 6500... | |
| [2023-02-24 08:07:48,093][795538] Num frames 6600... | |
| [2023-02-24 08:07:48,166][795538] Num frames 6700... | |
| [2023-02-24 08:07:48,237][795538] Num frames 6800... | |
| [2023-02-24 08:07:48,313][795538] Num frames 6900... | |
| [2023-02-24 08:07:48,383][795538] Num frames 7000... | |
| [2023-02-24 08:07:48,450][795538] Num frames 7100... | |
| [2023-02-24 08:07:48,516][795538] Num frames 7200... | |
| [2023-02-24 08:07:48,594][795538] Num frames 7300... | |
| [2023-02-24 08:07:48,669][795538] Num frames 7400... | |
| [2023-02-24 08:07:48,742][795538] Num frames 7500... | |
| [2023-02-24 08:07:48,814][795538] Num frames 7600... | |
| [2023-02-24 08:07:48,895][795538] Num frames 7700... | |
| [2023-02-24 08:07:48,996][795538] Avg episode rewards: #0: 21.345, true rewards: #0: 9.720 | |
| [2023-02-24 08:07:48,997][795538] Avg episode reward: 21.345, avg true_objective: 9.720 | |
| [2023-02-24 08:07:49,018][795538] Num frames 7800... | |
| [2023-02-24 08:07:49,092][795538] Num frames 7900... | |
| [2023-02-24 08:07:49,171][795538] Num frames 8000... | |
| [2023-02-24 08:07:49,241][795538] Num frames 8100... | |
| [2023-02-24 08:07:49,311][795538] Num frames 8200... | |
| [2023-02-24 08:07:49,424][795538] Avg episode rewards: #0: 20.098, true rewards: #0: 9.209 | |
| [2023-02-24 08:07:49,425][795538] Avg episode reward: 20.098, avg true_objective: 9.209 | |
| [2023-02-24 08:07:49,434][795538] Num frames 8300... | |
| [2023-02-24 08:07:49,502][795538] Num frames 8400... | |
| [2023-02-24 08:07:49,568][795538] Num frames 8500... | |
| [2023-02-24 08:07:49,643][795538] Num frames 8600... | |
| [2023-02-24 08:07:49,707][795538] Num frames 8700... | |
| [2023-02-24 08:07:49,763][795538] Avg episode rewards: #0: 18.704, true rewards: #0: 8.704 | |
| [2023-02-24 08:07:49,764][795538] Avg episode reward: 18.704, avg true_objective: 8.704 | |
| [2023-02-24 08:07:53,791][795538] Replay video saved to /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/replay.mp4! | |
| [2023-02-24 08:08:19,273][795538] The model has been pushed to https://huggingface.co/chqmatteo/rl_course_vizdoom_health_gathering_supreme | |
| [2023-02-24 08:08:42,140][795538] Environment doom_basic already registered, overwriting... | |
| [2023-02-24 08:08:42,141][795538] Environment doom_two_colors_easy already registered, overwriting... | |
| [2023-02-24 08:08:42,142][795538] Environment doom_two_colors_hard already registered, overwriting... | |
| [2023-02-24 08:08:42,142][795538] Environment doom_dm already registered, overwriting... | |
| [2023-02-24 08:08:42,142][795538] Environment doom_dwango5 already registered, overwriting... | |
| [2023-02-24 08:08:42,143][795538] Environment doom_my_way_home_flat_actions already registered, overwriting... | |
| [2023-02-24 08:08:42,143][795538] Environment doom_defend_the_center_flat_actions already registered, overwriting... | |
| [2023-02-24 08:08:42,143][795538] Environment doom_my_way_home already registered, overwriting... | |
| [2023-02-24 08:08:42,144][795538] Environment doom_deadly_corridor already registered, overwriting... | |
| [2023-02-24 08:08:42,144][795538] Environment doom_defend_the_center already registered, overwriting... | |
| [2023-02-24 08:08:42,144][795538] Environment doom_defend_the_line already registered, overwriting... | |
| [2023-02-24 08:08:42,145][795538] Environment doom_health_gathering already registered, overwriting... | |
| [2023-02-24 08:08:42,145][795538] Environment doom_health_gathering_supreme already registered, overwriting... | |
| [2023-02-24 08:08:42,145][795538] Environment doom_battle already registered, overwriting... | |
| [2023-02-24 08:08:42,146][795538] Environment doom_battle2 already registered, overwriting... | |
| [2023-02-24 08:08:42,146][795538] Environment doom_duel_bots already registered, overwriting... | |
| [2023-02-24 08:08:42,146][795538] Environment doom_deathmatch_bots already registered, overwriting... | |
| [2023-02-24 08:08:42,147][795538] Environment doom_duel already registered, overwriting... | |
| [2023-02-24 08:08:42,147][795538] Environment doom_deathmatch_full already registered, overwriting... | |
| [2023-02-24 08:08:42,148][795538] Environment doom_benchmark already registered, overwriting... | |
| [2023-02-24 08:08:42,148][795538] register_encoder_factory: <function make_vizdoom_encoder at 0x7efd09ae12d0> | |
| [2023-02-24 08:08:42,156][795538] Loading existing experiment configuration from /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/config.json | |
| [2023-02-24 08:08:42,157][795538] Experiment dir /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment already exists! | |
| [2023-02-24 08:08:42,158][795538] Resuming existing experiment from /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment... | |
| [2023-02-24 08:08:42,158][795538] Weights and Biases integration disabled | |
| [2023-02-24 08:08:42,159][795538] Environment var CUDA_VISIBLE_DEVICES is 1 | |
| [2023-02-24 08:08:43,053][795538] Starting experiment with the following configuration: | |
| help=False | |
| algo=APPO | |
| env=doom_health_gathering_supreme | |
| experiment=default_experiment | |
| train_dir=/mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir | |
| restart_behavior=resume | |
| device=gpu | |
| seed=None | |
| num_policies=1 | |
| async_rl=True | |
| serial_mode=False | |
| batched_sampling=False | |
| num_batches_to_accumulate=2 | |
| worker_num_splits=2 | |
| policy_workers_per_policy=1 | |
| max_policy_lag=1000 | |
| num_workers=8 | |
| num_envs_per_worker=4 | |
| batch_size=1024 | |
| num_batches_per_epoch=1 | |
| num_epochs=1 | |
| rollout=32 | |
| recurrence=32 | |
| shuffle_minibatches=False | |
| gamma=0.99 | |
| reward_scale=1.0 | |
| reward_clip=1000.0 | |
| value_bootstrap=False | |
| normalize_returns=True | |
| exploration_loss_coeff=0.001 | |
| value_loss_coeff=0.5 | |
| kl_loss_coeff=0.0 | |
| exploration_loss=symmetric_kl | |
| gae_lambda=0.95 | |
| ppo_clip_ratio=0.1 | |
| ppo_clip_value=0.2 | |
| with_vtrace=False | |
| vtrace_rho=1.0 | |
| vtrace_c=1.0 | |
| optimizer=adam | |
| adam_eps=1e-06 | |
| adam_beta1=0.9 | |
| adam_beta2=0.999 | |
| max_grad_norm=4.0 | |
| learning_rate=0.0001 | |
| lr_schedule=constant | |
| lr_schedule_kl_threshold=0.008 | |
| lr_adaptive_min=1e-06 | |
| lr_adaptive_max=0.01 | |
| obs_subtract_mean=0.0 | |
| obs_scale=255.0 | |
| normalize_input=True | |
| normalize_input_keys=None | |
| decorrelate_experience_max_seconds=0 | |
| decorrelate_envs_on_one_worker=True | |
| actor_worker_gpus=[] | |
| set_workers_cpu_affinity=True | |
| force_envs_single_thread=False | |
| default_niceness=0 | |
| log_to_file=True | |
| experiment_summaries_interval=10 | |
| flush_summaries_interval=30 | |
| stats_avg=100 | |
| summaries_use_frameskip=True | |
| heartbeat_interval=20 | |
| heartbeat_reporting_interval=600 | |
| train_for_env_steps=40000000 | |
| train_for_seconds=10000000000 | |
| save_every_sec=120 | |
| keep_checkpoints=2 | |
| load_checkpoint_kind=latest | |
| save_milestones_sec=-1 | |
| save_best_every_sec=5 | |
| save_best_metric=reward | |
| save_best_after=100000 | |
| benchmark=False | |
| encoder_mlp_layers=[512, 512] | |
| encoder_conv_architecture=convnet_simple | |
| encoder_conv_mlp_layers=[512] | |
| use_rnn=True | |
| rnn_size=512 | |
| rnn_type=gru | |
| rnn_num_layers=1 | |
| decoder_mlp_layers=[] | |
| nonlinearity=elu | |
| policy_initialization=orthogonal | |
| policy_init_gain=1.0 | |
| actor_critic_share_weights=True | |
| adaptive_stddev=True | |
| continuous_tanh_scale=0.0 | |
| initial_stddev=1.0 | |
| use_env_info_cache=False | |
| env_gpu_actions=False | |
| env_gpu_observations=True | |
| env_frameskip=4 | |
| env_framestack=1 | |
| pixel_format=CHW | |
| use_record_episode_statistics=False | |
| with_wandb=False | |
| wandb_user=None | |
| wandb_project=sample_factory | |
| wandb_group=None | |
| wandb_job_type=SF | |
| wandb_tags=[] | |
| with_pbt=False | |
| pbt_mix_policies_in_one_env=True | |
| pbt_period_env_steps=5000000 | |
| pbt_start_mutation=20000000 | |
| pbt_replace_fraction=0.3 | |
| pbt_mutation_rate=0.15 | |
| pbt_replace_reward_gap=0.1 | |
| pbt_replace_reward_gap_absolute=1e-06 | |
| pbt_optimize_gamma=False | |
| pbt_target_objective=true_objective | |
| pbt_perturb_min=1.1 | |
| pbt_perturb_max=1.5 | |
| num_agents=-1 | |
| num_humans=0 | |
| num_bots=-1 | |
| start_bot_difficulty=None | |
| timelimit=None | |
| res_w=128 | |
| res_h=72 | |
| wide_aspect_ratio=False | |
| eval_env_frameskip=1 | |
| fps=35 | |
| command_line=--env=doom_health_gathering_supreme --num_workers=8 --num_envs_per_worker=4 --train_for_env_steps=4000000 | |
| cli_args={'env': 'doom_health_gathering_supreme', 'num_workers': 8, 'num_envs_per_worker': 4, 'train_for_env_steps': 4000000} | |
| git_hash=1a2374cbd09490752b14aee6fdecfe64db411550 | |
| git_repo_name=https://github.com/huggingface/deep-rl-class.git | |
| [2023-02-24 08:08:43,053][795538] Saving configuration to /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/config.json... | |
| [2023-02-24 08:08:43,222][795538] Rollout worker 0 uses device cpu | |
| [2023-02-24 08:08:43,223][795538] Rollout worker 1 uses device cpu | |
| [2023-02-24 08:08:43,223][795538] Rollout worker 2 uses device cpu | |
| [2023-02-24 08:08:43,223][795538] Rollout worker 3 uses device cpu | |
| [2023-02-24 08:08:43,224][795538] Rollout worker 4 uses device cpu | |
| [2023-02-24 08:08:43,224][795538] Rollout worker 5 uses device cpu | |
| [2023-02-24 08:08:43,225][795538] Rollout worker 6 uses device cpu | |
| [2023-02-24 08:08:43,225][795538] Rollout worker 7 uses device cpu | |
| [2023-02-24 08:08:43,257][795538] Using GPUs [0] for process 0 (actually maps to GPUs [1]) | |
| [2023-02-24 08:08:43,258][795538] InferenceWorker_p0-w0: min num requests: 2 | |
| [2023-02-24 08:08:43,278][795538] Starting all processes... | |
| [2023-02-24 08:08:43,278][795538] Starting process learner_proc0 | |
| [2023-02-24 08:08:43,328][795538] Starting all processes... | |
| [2023-02-24 08:08:43,329][795538] Starting process inference_proc0-0 | |
| [2023-02-24 08:08:43,330][795538] Starting process rollout_proc0 | |
| [2023-02-24 08:08:43,330][795538] Starting process rollout_proc1 | |
| [2023-02-24 08:08:43,330][795538] Starting process rollout_proc2 | |
| [2023-02-24 08:08:43,331][795538] Starting process rollout_proc3 | |
| [2023-02-24 08:08:43,331][795538] Starting process rollout_proc4 | |
| [2023-02-24 08:08:43,331][795538] Starting process rollout_proc5 | |
| [2023-02-24 08:08:43,331][795538] Starting process rollout_proc6 | |
| [2023-02-24 08:08:43,331][795538] Starting process rollout_proc7 | |
| [2023-02-24 08:08:44,462][796098] Low niceness requires sudo! | |
| [2023-02-24 08:08:44,463][796098] Using GPUs [0] for process 0 (actually maps to GPUs [1]) | |
| [2023-02-24 08:08:44,463][796098] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [0]) for learning process 0 | |
| [2023-02-24 08:08:44,484][796098] Num visible devices: 1 | |
| [2023-02-24 08:08:44,522][796098] Starting seed is not provided | |
| [2023-02-24 08:08:44,522][796098] Using GPUs [0] for process 0 (actually maps to GPUs [1]) | |
| [2023-02-24 08:08:44,523][796098] Initializing actor-critic model on device cuda:0 | |
| [2023-02-24 08:08:44,523][796098] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-02-24 08:08:44,524][796098] RunningMeanStd input shape: (1,) | |
| [2023-02-24 08:08:44,549][796098] ConvEncoder: input_channels=3 | |
| [2023-02-24 08:08:44,706][796112] Low niceness requires sudo! | |
| [2023-02-24 08:08:44,707][796112] Using GPUs [0] for process 0 (actually maps to GPUs [1]) | |
| [2023-02-24 08:08:44,707][796112] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [0]) for inference process 0 | |
| [2023-02-24 08:08:44,724][796112] Num visible devices: 1 | |
| [2023-02-24 08:08:44,767][796113] Worker 2 uses CPU cores [2] | |
| [2023-02-24 08:08:44,802][796098] Conv encoder output size: 512 | |
| [2023-02-24 08:08:44,810][796098] Policy head output size: 512 | |
| [2023-02-24 08:08:44,842][796098] Created Actor Critic model with architecture: | |
| [2023-02-24 08:08:44,843][796098] ActorCriticSharedWeights( | |
| (obs_normalizer): ObservationNormalizer( | |
| (running_mean_std): RunningMeanStdDictInPlace( | |
| (running_mean_std): ModuleDict( | |
| (obs): RunningMeanStdInPlace() | |
| ) | |
| ) | |
| ) | |
| (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) | |
| (encoder): VizdoomEncoder( | |
| (basic_encoder): ConvEncoder( | |
| (enc): RecursiveScriptModule( | |
| original_name=ConvEncoderImpl | |
| (conv_head): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Conv2d) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| (2): RecursiveScriptModule(original_name=Conv2d) | |
| (3): RecursiveScriptModule(original_name=ELU) | |
| (4): RecursiveScriptModule(original_name=Conv2d) | |
| (5): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| (mlp_layers): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Linear) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| ) | |
| ) | |
| ) | |
| (core): ModelCoreRNN( | |
| (core): GRU(512, 512) | |
| ) | |
| (decoder): MlpDecoder( | |
| (mlp): Identity() | |
| ) | |
| (critic_linear): Linear(in_features=512, out_features=1, bias=True) | |
| (action_parameterization): ActionParameterizationDefault( | |
| (distribution_linear): Linear(in_features=512, out_features=5, bias=True) | |
| ) | |
| ) | |
| [2023-02-24 08:08:44,946][796119] Worker 6 uses CPU cores [6] | |
| [2023-02-24 08:08:44,948][796117] Worker 5 uses CPU cores [5] | |
| [2023-02-24 08:08:45,059][796114] Worker 1 uses CPU cores [1] | |
| [2023-02-24 08:08:45,108][796115] Worker 3 uses CPU cores [3] | |
| [2023-02-24 08:08:45,108][796111] Worker 0 uses CPU cores [0] | |
| [2023-02-24 08:08:45,114][796118] Worker 4 uses CPU cores [4] | |
| [2023-02-24 08:08:45,164][796116] Worker 7 uses CPU cores [7] | |
| [2023-02-24 08:08:46,945][796098] Using optimizer <class 'torch.optim.adam.Adam'> | |
| [2023-02-24 08:08:46,945][796098] Loading state from checkpoint /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000000403_1650688.pth... | |
| [2023-02-24 08:08:47,005][796098] Loading model from checkpoint | |
| [2023-02-24 08:08:47,007][796098] Loaded experiment state at self.train_step=403, self.env_steps=1650688 | |
| [2023-02-24 08:08:47,007][796098] Initialized policy 0 weights for model version 403 | |
| [2023-02-24 08:08:47,008][796098] LearnerWorker_p0 finished initialization! | |
| [2023-02-24 08:08:47,008][796098] Using GPUs [0] for process 0 (actually maps to GPUs [1]) | |
| [2023-02-24 08:08:47,160][795538] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 1650688. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2023-02-24 08:08:48,101][796112] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-02-24 08:08:48,101][796112] RunningMeanStd input shape: (1,) | |
| [2023-02-24 08:08:48,108][796112] ConvEncoder: input_channels=3 | |
| [2023-02-24 08:08:48,178][796112] Conv encoder output size: 512 | |
| [2023-02-24 08:08:48,178][796112] Policy head output size: 512 | |
| [2023-02-24 08:08:49,152][795538] Inference worker 0-0 is ready! | |
| [2023-02-24 08:08:49,152][795538] All inference workers are ready! Signal rollout workers to start! | |
| [2023-02-24 08:08:49,171][796116] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-24 08:08:49,171][796118] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-24 08:08:49,170][796115] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-24 08:08:49,171][796119] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-24 08:08:49,175][796111] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-24 08:08:49,176][796117] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-24 08:08:49,204][796113] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-24 08:08:49,205][796114] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-24 08:08:49,421][796115] Decorrelating experience for 0 frames... | |
| [2023-02-24 08:08:49,457][796116] Decorrelating experience for 0 frames... | |
| [2023-02-24 08:08:49,467][796114] Decorrelating experience for 0 frames... | |
| [2023-02-24 08:08:49,471][796118] Decorrelating experience for 0 frames... | |
| [2023-02-24 08:08:49,475][796117] Decorrelating experience for 0 frames... | |
| [2023-02-24 08:08:49,653][796115] Decorrelating experience for 32 frames... | |
| [2023-02-24 08:08:49,732][796116] Decorrelating experience for 32 frames... | |
| [2023-02-24 08:08:49,752][796118] Decorrelating experience for 32 frames... | |
| [2023-02-24 08:08:49,758][796117] Decorrelating experience for 32 frames... | |
| [2023-02-24 08:08:49,795][796119] Decorrelating experience for 0 frames... | |
| [2023-02-24 08:08:49,932][796114] Decorrelating experience for 32 frames... | |
| [2023-02-24 08:08:50,017][796111] Decorrelating experience for 0 frames... | |
| [2023-02-24 08:08:50,033][796116] Decorrelating experience for 64 frames... | |
| [2023-02-24 08:08:50,071][796119] Decorrelating experience for 32 frames... | |
| [2023-02-24 08:08:50,078][796115] Decorrelating experience for 64 frames... | |
| [2023-02-24 08:08:50,109][796113] Decorrelating experience for 0 frames... | |
| [2023-02-24 08:08:50,145][796118] Decorrelating experience for 64 frames... | |
| [2023-02-24 08:08:50,241][796111] Decorrelating experience for 32 frames... | |
| [2023-02-24 08:08:50,325][796116] Decorrelating experience for 96 frames... | |
| [2023-02-24 08:08:50,338][796115] Decorrelating experience for 96 frames... | |
| [2023-02-24 08:08:50,377][796117] Decorrelating experience for 64 frames... | |
| [2023-02-24 08:08:50,445][796118] Decorrelating experience for 96 frames... | |
| [2023-02-24 08:08:50,502][796111] Decorrelating experience for 64 frames... | |
| [2023-02-24 08:08:50,527][796119] Decorrelating experience for 64 frames... | |
| [2023-02-24 08:08:50,669][796117] Decorrelating experience for 96 frames... | |
| [2023-02-24 08:08:50,704][796113] Decorrelating experience for 32 frames... | |
| [2023-02-24 08:08:50,716][796114] Decorrelating experience for 64 frames... | |
| [2023-02-24 08:08:50,804][796111] Decorrelating experience for 96 frames... | |
| [2023-02-24 08:08:50,833][796119] Decorrelating experience for 96 frames... | |
| [2023-02-24 08:08:50,958][796113] Decorrelating experience for 64 frames... | |
| [2023-02-24 08:08:50,977][796114] Decorrelating experience for 96 frames... | |
| [2023-02-24 08:08:51,210][796113] Decorrelating experience for 96 frames... | |
| [2023-02-24 08:08:52,160][795538] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 1650688. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2023-02-24 08:08:52,161][795538] Avg episode reward: [(0, '0.903')] | |
| [2023-02-24 08:08:52,586][796098] Signal inference workers to stop experience collection... | |
| [2023-02-24 08:08:52,593][796112] InferenceWorker_p0-w0: stopping experience collection | |
| [2023-02-24 08:08:54,537][796098] Signal inference workers to resume experience collection... | |
| [2023-02-24 08:08:54,538][796112] InferenceWorker_p0-w0: resuming experience collection | |
| [2023-02-24 08:08:57,090][796112] Updated weights for policy 0, policy_version 413 (0.0247) | |
| [2023-02-24 08:08:57,160][795538] Fps is (10 sec: 4096.0, 60 sec: 4096.0, 300 sec: 4096.0). Total num frames: 1691648. Throughput: 0: 332.8. Samples: 3328. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2023-02-24 08:08:57,161][795538] Avg episode reward: [(0, '8.033')] | |
| [2023-02-24 08:08:59,657][796112] Updated weights for policy 0, policy_version 423 (0.0013) | |
| [2023-02-24 08:09:02,160][795538] Fps is (10 sec: 11878.4, 60 sec: 7918.9, 300 sec: 7918.9). Total num frames: 1769472. Throughput: 0: 1818.4. Samples: 27276. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-02-24 08:09:02,161][795538] Avg episode reward: [(0, '21.201')] | |
| [2023-02-24 08:09:02,163][796098] Saving new best policy, reward=21.201! | |
| [2023-02-24 08:09:02,326][796112] Updated weights for policy 0, policy_version 433 (0.0009) | |
| [2023-02-24 08:09:03,254][795538] Heartbeat connected on LearnerWorker_p0 | |
| [2023-02-24 08:09:03,262][795538] Heartbeat connected on InferenceWorker_p0-w0 | |
| [2023-02-24 08:09:03,263][795538] Heartbeat connected on Batcher_0 | |
| [2023-02-24 08:09:03,264][795538] Heartbeat connected on RolloutWorker_w1 | |
| [2023-02-24 08:09:03,266][795538] Heartbeat connected on RolloutWorker_w2 | |
| [2023-02-24 08:09:03,269][795538] Heartbeat connected on RolloutWorker_w3 | |
| [2023-02-24 08:09:03,271][795538] Heartbeat connected on RolloutWorker_w0 | |
| [2023-02-24 08:09:03,275][795538] Heartbeat connected on RolloutWorker_w5 | |
| [2023-02-24 08:09:03,276][795538] Heartbeat connected on RolloutWorker_w4 | |
| [2023-02-24 08:09:03,280][795538] Heartbeat connected on RolloutWorker_w7 | |
| [2023-02-24 08:09:03,282][795538] Heartbeat connected on RolloutWorker_w6 | |
| [2023-02-24 08:09:04,840][796112] Updated weights for policy 0, policy_version 443 (0.0007) | |
| [2023-02-24 08:09:07,160][795538] Fps is (10 sec: 15974.5, 60 sec: 10035.2, 300 sec: 10035.2). Total num frames: 1851392. Throughput: 0: 2569.2. Samples: 51384. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-02-24 08:09:07,161][795538] Avg episode reward: [(0, '18.568')] | |
| [2023-02-24 08:09:07,343][796112] Updated weights for policy 0, policy_version 453 (0.0007) | |
| [2023-02-24 08:09:09,849][796112] Updated weights for policy 0, policy_version 463 (0.0006) | |
| [2023-02-24 08:09:12,160][795538] Fps is (10 sec: 16384.0, 60 sec: 11304.9, 300 sec: 11304.9). Total num frames: 1933312. Throughput: 0: 2543.8. Samples: 63594. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-02-24 08:09:12,161][795538] Avg episode reward: [(0, '20.173')] | |
| [2023-02-24 08:09:12,344][796112] Updated weights for policy 0, policy_version 473 (0.0006) | |
| [2023-02-24 08:09:14,882][796112] Updated weights for policy 0, policy_version 483 (0.0007) | |
| [2023-02-24 08:09:17,160][795538] Fps is (10 sec: 16384.0, 60 sec: 12151.5, 300 sec: 12151.5). Total num frames: 2015232. Throughput: 0: 2934.7. Samples: 88042. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-02-24 08:09:17,161][795538] Avg episode reward: [(0, '21.495')] | |
| [2023-02-24 08:09:17,162][796098] Saving new best policy, reward=21.495! | |
| [2023-02-24 08:09:17,416][796112] Updated weights for policy 0, policy_version 493 (0.0009) | |
| [2023-02-24 08:09:19,892][796112] Updated weights for policy 0, policy_version 503 (0.0006) | |
| [2023-02-24 08:09:22,160][795538] Fps is (10 sec: 16384.0, 60 sec: 12756.1, 300 sec: 12756.1). Total num frames: 2097152. Throughput: 0: 3214.6. Samples: 112512. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-24 08:09:22,161][795538] Avg episode reward: [(0, '18.695')] | |
| [2023-02-24 08:09:22,364][796112] Updated weights for policy 0, policy_version 513 (0.0010) | |
| [2023-02-24 08:09:24,860][796112] Updated weights for policy 0, policy_version 523 (0.0007) | |
| [2023-02-24 08:09:27,160][795538] Fps is (10 sec: 16384.1, 60 sec: 13209.6, 300 sec: 13209.6). Total num frames: 2179072. Throughput: 0: 3120.8. Samples: 124832. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-02-24 08:09:27,161][795538] Avg episode reward: [(0, '21.156')] | |
| [2023-02-24 08:09:27,388][796112] Updated weights for policy 0, policy_version 533 (0.0006) | |
| [2023-02-24 08:09:29,933][796112] Updated weights for policy 0, policy_version 543 (0.0011) | |
| [2023-02-24 08:09:32,160][795538] Fps is (10 sec: 15974.5, 60 sec: 13471.3, 300 sec: 13471.3). Total num frames: 2256896. Throughput: 0: 3315.7. Samples: 149206. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-02-24 08:09:32,164][795538] Avg episode reward: [(0, '19.110')] | |
| [2023-02-24 08:09:32,491][796112] Updated weights for policy 0, policy_version 553 (0.0008) | |
| [2023-02-24 08:09:34,962][796112] Updated weights for policy 0, policy_version 563 (0.0006) | |
| [2023-02-24 08:09:37,160][795538] Fps is (10 sec: 15974.4, 60 sec: 13762.6, 300 sec: 13762.6). Total num frames: 2338816. Throughput: 0: 3865.1. Samples: 173928. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-02-24 08:09:37,160][795538] Avg episode reward: [(0, '20.430')] | |
| [2023-02-24 08:09:37,443][796112] Updated weights for policy 0, policy_version 573 (0.0006) | |
| [2023-02-24 08:09:40,104][796112] Updated weights for policy 0, policy_version 583 (0.0007) | |
| [2023-02-24 08:09:42,160][795538] Fps is (10 sec: 15974.4, 60 sec: 13926.4, 300 sec: 13926.4). Total num frames: 2416640. Throughput: 0: 4050.8. Samples: 185616. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) | |
| [2023-02-24 08:09:42,160][795538] Avg episode reward: [(0, '22.387')] | |
| [2023-02-24 08:09:42,172][796098] Saving new best policy, reward=22.387! | |
| [2023-02-24 08:09:42,735][796112] Updated weights for policy 0, policy_version 593 (0.0008) | |
| [2023-02-24 08:09:45,371][796112] Updated weights for policy 0, policy_version 603 (0.0006) | |
| [2023-02-24 08:09:47,160][795538] Fps is (10 sec: 15974.3, 60 sec: 14131.2, 300 sec: 14131.2). Total num frames: 2498560. Throughput: 0: 4036.7. Samples: 208926. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) | |
| [2023-02-24 08:09:47,162][795538] Avg episode reward: [(0, '23.294')] | |
| [2023-02-24 08:09:47,163][796098] Saving new best policy, reward=23.294! | |
| [2023-02-24 08:09:47,936][796112] Updated weights for policy 0, policy_version 613 (0.0009) | |
| [2023-02-24 08:09:50,471][796112] Updated weights for policy 0, policy_version 623 (0.0006) | |
| [2023-02-24 08:09:52,160][795538] Fps is (10 sec: 15974.4, 60 sec: 15428.3, 300 sec: 14241.5). Total num frames: 2576384. Throughput: 0: 4030.0. Samples: 232734. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2023-02-24 08:09:52,161][795538] Avg episode reward: [(0, '23.526')] | |
| [2023-02-24 08:09:52,164][796098] Saving new best policy, reward=23.526! | |
| [2023-02-24 08:09:53,087][796112] Updated weights for policy 0, policy_version 633 (0.0006) | |
| [2023-02-24 08:09:55,707][796112] Updated weights for policy 0, policy_version 643 (0.0007) | |
| [2023-02-24 08:09:57,160][795538] Fps is (10 sec: 15564.8, 60 sec: 16042.7, 300 sec: 14336.0). Total num frames: 2654208. Throughput: 0: 4017.6. Samples: 244384. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-02-24 08:09:57,161][795538] Avg episode reward: [(0, '23.324')] | |
| [2023-02-24 08:09:58,316][796112] Updated weights for policy 0, policy_version 653 (0.0008) | |
| [2023-02-24 08:10:00,905][796112] Updated weights for policy 0, policy_version 663 (0.0006) | |
| [2023-02-24 08:10:02,160][795538] Fps is (10 sec: 15564.8, 60 sec: 16042.7, 300 sec: 14417.9). Total num frames: 2732032. Throughput: 0: 4002.8. Samples: 268166. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-02-24 08:10:02,161][795538] Avg episode reward: [(0, '23.305')] | |
| [2023-02-24 08:10:03,498][796112] Updated weights for policy 0, policy_version 673 (0.0006) | |
| [2023-02-24 08:10:06,102][796112] Updated weights for policy 0, policy_version 683 (0.0006) | |
| [2023-02-24 08:10:07,160][795538] Fps is (10 sec: 15564.8, 60 sec: 15974.4, 300 sec: 14489.6). Total num frames: 2809856. Throughput: 0: 3723.8. Samples: 280082. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) | |
| [2023-02-24 08:10:07,162][795538] Avg episode reward: [(0, '24.939')] | |
| [2023-02-24 08:10:07,168][796098] Saving new best policy, reward=24.939! | |
| [2023-02-24 08:10:08,749][796112] Updated weights for policy 0, policy_version 693 (0.0006) | |
| [2023-02-24 08:10:11,316][796112] Updated weights for policy 0, policy_version 703 (0.0007) | |
| [2023-02-24 08:10:12,160][795538] Fps is (10 sec: 15974.4, 60 sec: 15974.4, 300 sec: 14601.0). Total num frames: 2891776. Throughput: 0: 3972.7. Samples: 303606. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-02-24 08:10:12,161][795538] Avg episode reward: [(0, '27.390')] | |
| [2023-02-24 08:10:12,165][796098] Saving new best policy, reward=27.390! | |
| [2023-02-24 08:10:13,931][796112] Updated weights for policy 0, policy_version 713 (0.0006) | |
| [2023-02-24 08:10:16,517][796112] Updated weights for policy 0, policy_version 723 (0.0007) | |
| [2023-02-24 08:10:17,160][795538] Fps is (10 sec: 15974.5, 60 sec: 15906.1, 300 sec: 14654.6). Total num frames: 2969600. Throughput: 0: 3957.6. Samples: 327298. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-02-24 08:10:17,160][795538] Avg episode reward: [(0, '23.654')] | |
| [2023-02-24 08:10:19,064][796112] Updated weights for policy 0, policy_version 733 (0.0006) | |
| [2023-02-24 08:10:21,654][796112] Updated weights for policy 0, policy_version 743 (0.0007) | |
| [2023-02-24 08:10:22,160][795538] Fps is (10 sec: 15564.9, 60 sec: 15837.9, 300 sec: 14702.5). Total num frames: 3047424. Throughput: 0: 3672.8. Samples: 339206. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-02-24 08:10:22,161][795538] Avg episode reward: [(0, '23.892')] | |
| [2023-02-24 08:10:24,239][796112] Updated weights for policy 0, policy_version 753 (0.0007) | |
| [2023-02-24 08:10:26,790][796112] Updated weights for policy 0, policy_version 763 (0.0007) | |
| [2023-02-24 08:10:27,160][795538] Fps is (10 sec: 15974.3, 60 sec: 15837.8, 300 sec: 14786.6). Total num frames: 3129344. Throughput: 0: 3942.0. Samples: 363004. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-02-24 08:10:27,161][795538] Avg episode reward: [(0, '24.204')] | |
| [2023-02-24 08:10:29,329][796112] Updated weights for policy 0, policy_version 773 (0.0007) | |
| [2023-02-24 08:10:31,939][796112] Updated weights for policy 0, policy_version 783 (0.0006) | |
| [2023-02-24 08:10:32,160][795538] Fps is (10 sec: 15974.3, 60 sec: 15837.9, 300 sec: 14823.6). Total num frames: 3207168. Throughput: 0: 3952.3. Samples: 386780. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-02-24 08:10:32,161][795538] Avg episode reward: [(0, '26.121')] | |
| [2023-02-24 08:10:34,527][796112] Updated weights for policy 0, policy_version 793 (0.0006) | |
| [2023-02-24 08:10:37,160][796112] Updated weights for policy 0, policy_version 803 (0.0006) | |
| [2023-02-24 08:10:37,160][795538] Fps is (10 sec: 15974.5, 60 sec: 15837.9, 300 sec: 14894.6). Total num frames: 3289088. Throughput: 0: 3950.8. Samples: 410518. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-02-24 08:10:37,161][795538] Avg episode reward: [(0, '24.946')] | |
| [2023-02-24 08:10:39,753][796112] Updated weights for policy 0, policy_version 813 (0.0007) | |
| [2023-02-24 08:10:42,160][795538] Fps is (10 sec: 15974.4, 60 sec: 15837.9, 300 sec: 14923.7). Total num frames: 3366912. Throughput: 0: 3957.0. Samples: 422448. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-02-24 08:10:42,160][795538] Avg episode reward: [(0, '24.684')] | |
| [2023-02-24 08:10:42,164][796098] Saving /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000000822_3366912.pth... | |
| [2023-02-24 08:10:42,260][796098] Removing /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000000268_1097728.pth | |
| [2023-02-24 08:10:42,367][796112] Updated weights for policy 0, policy_version 823 (0.0006) | |
| [2023-02-24 08:10:44,973][796112] Updated weights for policy 0, policy_version 833 (0.0006) | |
| [2023-02-24 08:10:47,160][795538] Fps is (10 sec: 15564.7, 60 sec: 15769.6, 300 sec: 14950.4). Total num frames: 3444736. Throughput: 0: 3949.0. Samples: 445872. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) | |
| [2023-02-24 08:10:47,160][795538] Avg episode reward: [(0, '26.184')] | |
| [2023-02-24 08:10:47,617][796112] Updated weights for policy 0, policy_version 843 (0.0006) | |
| [2023-02-24 08:10:50,205][796112] Updated weights for policy 0, policy_version 853 (0.0007) | |
| [2023-02-24 08:10:52,160][795538] Fps is (10 sec: 15564.9, 60 sec: 15769.6, 300 sec: 14975.0). Total num frames: 3522560. Throughput: 0: 4210.3. Samples: 469546. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-02-24 08:10:52,161][795538] Avg episode reward: [(0, '26.862')] | |
| [2023-02-24 08:10:52,756][796112] Updated weights for policy 0, policy_version 863 (0.0008) | |
| [2023-02-24 08:10:55,295][796112] Updated weights for policy 0, policy_version 873 (0.0006) | |
| [2023-02-24 08:10:57,160][795538] Fps is (10 sec: 15974.4, 60 sec: 15837.9, 300 sec: 15029.2). Total num frames: 3604480. Throughput: 0: 3953.6. Samples: 481518. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) | |
| [2023-02-24 08:10:57,161][795538] Avg episode reward: [(0, '25.390')] | |
| [2023-02-24 08:10:57,895][796112] Updated weights for policy 0, policy_version 883 (0.0007) | |
| [2023-02-24 08:11:00,550][796112] Updated weights for policy 0, policy_version 893 (0.0009) | |
| [2023-02-24 08:11:02,160][795538] Fps is (10 sec: 15974.2, 60 sec: 15837.8, 300 sec: 15049.0). Total num frames: 3682304. Throughput: 0: 3949.9. Samples: 505042. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-02-24 08:11:02,161][795538] Avg episode reward: [(0, '25.427')] | |
| [2023-02-24 08:11:03,150][796112] Updated weights for policy 0, policy_version 903 (0.0008) | |
| [2023-02-24 08:11:05,738][796112] Updated weights for policy 0, policy_version 913 (0.0008) | |
| [2023-02-24 08:11:07,160][795538] Fps is (10 sec: 15564.9, 60 sec: 15837.9, 300 sec: 15067.4). Total num frames: 3760128. Throughput: 0: 4217.3. Samples: 528986. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-02-24 08:11:07,161][795538] Avg episode reward: [(0, '25.526')] | |
| [2023-02-24 08:11:08,213][796112] Updated weights for policy 0, policy_version 923 (0.0006) | |
| [2023-02-24 08:11:10,734][796112] Updated weights for policy 0, policy_version 933 (0.0007) | |
| [2023-02-24 08:11:12,160][795538] Fps is (10 sec: 15974.6, 60 sec: 15837.9, 300 sec: 15112.8). Total num frames: 3842048. Throughput: 0: 3965.5. Samples: 541452. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-02-24 08:11:12,160][795538] Avg episode reward: [(0, '26.175')] | |
| [2023-02-24 08:11:13,377][796112] Updated weights for policy 0, policy_version 943 (0.0007) | |
| [2023-02-24 08:11:15,931][796112] Updated weights for policy 0, policy_version 953 (0.0007) | |
| [2023-02-24 08:11:17,160][795538] Fps is (10 sec: 15974.4, 60 sec: 15837.9, 300 sec: 15127.9). Total num frames: 3919872. Throughput: 0: 3960.2. Samples: 564990. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-02-24 08:11:17,161][795538] Avg episode reward: [(0, '26.638')] | |
| [2023-02-24 08:11:18,529][796112] Updated weights for policy 0, policy_version 963 (0.0008) | |
| [2023-02-24 08:11:21,078][796112] Updated weights for policy 0, policy_version 973 (0.0007) | |
| [2023-02-24 08:11:22,160][795538] Fps is (10 sec: 15974.3, 60 sec: 15906.1, 300 sec: 15168.4). Total num frames: 4001792. Throughput: 0: 3964.5. Samples: 588920. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-02-24 08:11:22,161][795538] Avg episode reward: [(0, '26.420')] | |
| [2023-02-24 08:11:23,650][796112] Updated weights for policy 0, policy_version 983 (0.0006) | |
| [2023-02-24 08:11:26,246][796112] Updated weights for policy 0, policy_version 993 (0.0006) | |
| [2023-02-24 08:11:27,160][795538] Fps is (10 sec: 15974.4, 60 sec: 15837.9, 300 sec: 15180.8). Total num frames: 4079616. Throughput: 0: 3963.6. Samples: 600810. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-02-24 08:11:27,161][795538] Avg episode reward: [(0, '23.763')] | |
| [2023-02-24 08:11:28,856][796112] Updated weights for policy 0, policy_version 1003 (0.0006) | |
| [2023-02-24 08:11:31,443][796112] Updated weights for policy 0, policy_version 1013 (0.0006) | |
| [2023-02-24 08:11:32,160][795538] Fps is (10 sec: 15564.8, 60 sec: 15837.9, 300 sec: 15192.4). Total num frames: 4157440. Throughput: 0: 3970.3. Samples: 624536. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) | |
| [2023-02-24 08:11:32,161][795538] Avg episode reward: [(0, '29.275')] | |
| [2023-02-24 08:11:32,165][796098] Saving new best policy, reward=29.275! | |
| [2023-02-24 08:11:34,059][796112] Updated weights for policy 0, policy_version 1023 (0.0007) | |
| [2023-02-24 08:11:36,631][796112] Updated weights for policy 0, policy_version 1033 (0.0006) | |
| [2023-02-24 08:11:37,160][795538] Fps is (10 sec: 15974.4, 60 sec: 15837.9, 300 sec: 15227.5). Total num frames: 4239360. Throughput: 0: 3966.3. Samples: 648028. Policy #0 lag: (min: 0.0, avg: 0.8, max: 1.0) | |
| [2023-02-24 08:11:37,161][795538] Avg episode reward: [(0, '30.081')] | |
| [2023-02-24 08:11:37,162][796098] Saving new best policy, reward=30.081! | |
| [2023-02-24 08:11:39,264][796112] Updated weights for policy 0, policy_version 1043 (0.0006) | |
| [2023-02-24 08:11:41,882][796112] Updated weights for policy 0, policy_version 1053 (0.0008) | |
| [2023-02-24 08:11:42,160][795538] Fps is (10 sec: 15974.4, 60 sec: 15837.9, 300 sec: 15237.1). Total num frames: 4317184. Throughput: 0: 3961.7. Samples: 659794. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-02-24 08:11:42,162][795538] Avg episode reward: [(0, '28.237')] | |
| [2023-02-24 08:11:44,465][796112] Updated weights for policy 0, policy_version 1063 (0.0007) | |
| [2023-02-24 08:11:47,078][796112] Updated weights for policy 0, policy_version 1073 (0.0007) | |
| [2023-02-24 08:11:47,160][795538] Fps is (10 sec: 15564.9, 60 sec: 15837.9, 300 sec: 15246.2). Total num frames: 4395008. Throughput: 0: 3964.2. Samples: 683430. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-02-24 08:11:47,160][795538] Avg episode reward: [(0, '27.334')] | |
| [2023-02-24 08:11:49,667][796112] Updated weights for policy 0, policy_version 1083 (0.0007) | |
| [2023-02-24 08:11:52,160][795538] Fps is (10 sec: 15564.8, 60 sec: 15837.9, 300 sec: 15254.8). Total num frames: 4472832. Throughput: 0: 3957.7. Samples: 707084. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) | |
| [2023-02-24 08:11:52,161][795538] Avg episode reward: [(0, '27.060')] | |
| [2023-02-24 08:11:52,282][796112] Updated weights for policy 0, policy_version 1093 (0.0006) | |
| [2023-02-24 08:11:54,877][796112] Updated weights for policy 0, policy_version 1103 (0.0006) | |
| [2023-02-24 08:11:57,160][795538] Fps is (10 sec: 15564.7, 60 sec: 15769.6, 300 sec: 15263.0). Total num frames: 4550656. Throughput: 0: 3941.8. Samples: 718832. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-02-24 08:11:57,161][795538] Avg episode reward: [(0, '26.759')] | |
| [2023-02-24 08:11:57,461][796112] Updated weights for policy 0, policy_version 1113 (0.0006) | |
| [2023-02-24 08:12:00,051][796112] Updated weights for policy 0, policy_version 1123 (0.0006) | |
| [2023-02-24 08:12:02,160][795538] Fps is (10 sec: 15974.5, 60 sec: 15837.9, 300 sec: 15291.7). Total num frames: 4632576. Throughput: 0: 3946.0. Samples: 742562. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-02-24 08:12:02,161][795538] Avg episode reward: [(0, '31.137')] | |
| [2023-02-24 08:12:02,164][796098] Saving new best policy, reward=31.137! | |
| [2023-02-24 08:12:02,698][796112] Updated weights for policy 0, policy_version 1133 (0.0008) | |
| [2023-02-24 08:12:05,265][796112] Updated weights for policy 0, policy_version 1143 (0.0007) | |
| [2023-02-24 08:12:07,160][795538] Fps is (10 sec: 15974.4, 60 sec: 15837.9, 300 sec: 15298.6). Total num frames: 4710400. Throughput: 0: 3936.4. Samples: 766056. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-02-24 08:12:07,161][795538] Avg episode reward: [(0, '29.446')] | |
| [2023-02-24 08:12:07,854][796112] Updated weights for policy 0, policy_version 1153 (0.0009) | |
| [2023-02-24 08:12:10,527][796112] Updated weights for policy 0, policy_version 1163 (0.0008) | |
| [2023-02-24 08:12:12,160][795538] Fps is (10 sec: 15564.8, 60 sec: 15769.6, 300 sec: 15305.1). Total num frames: 4788224. Throughput: 0: 3933.4. Samples: 777814. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-02-24 08:12:12,161][795538] Avg episode reward: [(0, '25.534')] | |
| [2023-02-24 08:12:13,157][796112] Updated weights for policy 0, policy_version 1173 (0.0008) | |
| [2023-02-24 08:12:15,767][796112] Updated weights for policy 0, policy_version 1183 (0.0007) | |
| [2023-02-24 08:12:17,160][795538] Fps is (10 sec: 15564.5, 60 sec: 15769.6, 300 sec: 15311.2). Total num frames: 4866048. Throughput: 0: 3927.5. Samples: 801272. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-02-24 08:12:17,161][795538] Avg episode reward: [(0, '28.476')] | |
| [2023-02-24 08:12:18,390][796112] Updated weights for policy 0, policy_version 1193 (0.0006) | |
| [2023-02-24 08:12:21,010][796112] Updated weights for policy 0, policy_version 1203 (0.0008) | |
| [2023-02-24 08:12:22,161][795538] Fps is (10 sec: 15563.3, 60 sec: 15701.1, 300 sec: 15317.1). Total num frames: 4943872. Throughput: 0: 3926.3. Samples: 824714. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) | |
| [2023-02-24 08:12:22,162][795538] Avg episode reward: [(0, '29.026')] | |
| [2023-02-24 08:12:23,640][796112] Updated weights for policy 0, policy_version 1213 (0.0007) | |
| [2023-02-24 08:12:26,277][796112] Updated weights for policy 0, policy_version 1223 (0.0006) | |
| [2023-02-24 08:12:27,160][795538] Fps is (10 sec: 15565.1, 60 sec: 15701.3, 300 sec: 15322.8). Total num frames: 5021696. Throughput: 0: 3925.6. Samples: 836444. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) | |
| [2023-02-24 08:12:27,161][795538] Avg episode reward: [(0, '28.968')] | |
| [2023-02-24 08:12:28,839][796112] Updated weights for policy 0, policy_version 1233 (0.0008) | |
| [2023-02-24 08:12:31,424][796112] Updated weights for policy 0, policy_version 1243 (0.0009) | |
| [2023-02-24 08:12:32,160][795538] Fps is (10 sec: 15566.2, 60 sec: 15701.3, 300 sec: 15328.1). Total num frames: 5099520. Throughput: 0: 3926.8. Samples: 860134. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) | |
| [2023-02-24 08:12:32,161][795538] Avg episode reward: [(0, '30.627')] | |
| [2023-02-24 08:12:34,002][796112] Updated weights for policy 0, policy_version 1253 (0.0006) | |
| [2023-02-24 08:12:36,598][796112] Updated weights for policy 0, policy_version 1263 (0.0007) | |
| [2023-02-24 08:12:37,160][795538] Fps is (10 sec: 15974.5, 60 sec: 15701.3, 300 sec: 15351.1). Total num frames: 5181440. Throughput: 0: 3927.4. Samples: 883818. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-02-24 08:12:37,160][795538] Avg episode reward: [(0, '30.119')] | |
| [2023-02-24 08:12:39,183][796112] Updated weights for policy 0, policy_version 1273 (0.0006) | |
| [2023-02-24 08:12:41,827][796112] Updated weights for policy 0, policy_version 1283 (0.0006) | |
| [2023-02-24 08:12:42,160][795538] Fps is (10 sec: 15974.3, 60 sec: 15701.3, 300 sec: 15355.6). Total num frames: 5259264. Throughput: 0: 3932.7. Samples: 895804. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) | |
| [2023-02-24 08:12:42,161][795538] Avg episode reward: [(0, '28.016')] | |
| [2023-02-24 08:12:42,165][796098] Saving /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000001284_5259264.pth... | |
| [2023-02-24 08:12:42,261][796098] Removing /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000000403_1650688.pth | |
| [2023-02-24 08:12:44,477][796112] Updated weights for policy 0, policy_version 1293 (0.0006) | |
| [2023-02-24 08:12:47,112][796112] Updated weights for policy 0, policy_version 1303 (0.0007) | |
| [2023-02-24 08:12:47,160][795538] Fps is (10 sec: 15564.7, 60 sec: 15701.3, 300 sec: 15360.0). Total num frames: 5337088. Throughput: 0: 3918.7. Samples: 918904. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) | |
| [2023-02-24 08:12:47,161][795538] Avg episode reward: [(0, '29.296')] | |
| [2023-02-24 08:12:49,686][796112] Updated weights for policy 0, policy_version 1313 (0.0006) | |
| [2023-02-24 08:12:52,160][795538] Fps is (10 sec: 15564.8, 60 sec: 15701.3, 300 sec: 15364.2). Total num frames: 5414912. Throughput: 0: 3658.7. Samples: 930696. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-02-24 08:12:52,161][795538] Avg episode reward: [(0, '29.969')] | |
| [2023-02-24 08:12:52,236][796112] Updated weights for policy 0, policy_version 1323 (0.0007) | |
| [2023-02-24 08:12:54,849][796112] Updated weights for policy 0, policy_version 1333 (0.0006) | |
| [2023-02-24 08:12:57,160][795538] Fps is (10 sec: 15564.9, 60 sec: 15701.3, 300 sec: 15368.2). Total num frames: 5492736. Throughput: 0: 3924.4. Samples: 954414. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-02-24 08:12:57,160][795538] Avg episode reward: [(0, '26.997')] | |
| [2023-02-24 08:12:57,451][796112] Updated weights for policy 0, policy_version 1343 (0.0007) | |
| [2023-02-24 08:13:00,062][796112] Updated weights for policy 0, policy_version 1353 (0.0007) | |
| [2023-02-24 08:13:02,127][795538] Keyboard interrupt detected in the event loop EvtLoop [Runner_EvtLoop, process=main process 795538], exiting... | |
| [2023-02-24 08:13:02,128][796098] Stopping Batcher_0... | |
| [2023-02-24 08:13:02,129][796098] Loop batcher_evt_loop terminating... | |
| [2023-02-24 08:13:02,130][796098] Saving /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000001361_5574656.pth... | |
| [2023-02-24 08:13:02,130][795538] Runner profile tree view: | |
| main_loop: 258.8521 | |
| [2023-02-24 08:13:02,130][795538] Collected {0: 5574656}, FPS: 15159.1 | |
| [2023-02-24 08:13:02,136][796113] Stopping RolloutWorker_w2... | |
| [2023-02-24 08:13:02,137][796113] Loop rollout_proc2_evt_loop terminating... | |
| [2023-02-24 08:13:02,139][796118] Stopping RolloutWorker_w4... | |
| [2023-02-24 08:13:02,139][796118] Loop rollout_proc4_evt_loop terminating... | |
| [2023-02-24 08:13:02,139][796117] Stopping RolloutWorker_w5... | |
| [2023-02-24 08:13:02,140][796117] Loop rollout_proc5_evt_loop terminating... | |
| [2023-02-24 08:13:02,142][796115] Stopping RolloutWorker_w3... | |
| [2023-02-24 08:13:02,143][796115] Loop rollout_proc3_evt_loop terminating... | |
| [2023-02-24 08:13:02,143][796111] Stopping RolloutWorker_w0... | |
| [2023-02-24 08:13:02,144][796119] Stopping RolloutWorker_w6... | |
| [2023-02-24 08:13:02,144][796111] Loop rollout_proc0_evt_loop terminating... | |
| [2023-02-24 08:13:02,144][796119] Loop rollout_proc6_evt_loop terminating... | |
| [2023-02-24 08:13:02,148][796114] Stopping RolloutWorker_w1... | |
| [2023-02-24 08:13:02,148][796114] Loop rollout_proc1_evt_loop terminating... | |
| [2023-02-24 08:13:02,151][796116] Stopping RolloutWorker_w7... | |
| [2023-02-24 08:13:02,152][796116] Loop rollout_proc7_evt_loop terminating... | |
| [2023-02-24 08:13:02,163][796112] Weights refcount: 2 0 | |
| [2023-02-24 08:13:02,171][796112] Stopping InferenceWorker_p0-w0... | |
| [2023-02-24 08:13:02,172][796112] Loop inference_proc0-0_evt_loop terminating... | |
| [2023-02-24 08:13:02,301][796098] Removing /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000000822_3366912.pth | |
| [2023-02-24 08:13:02,305][796098] Stopping LearnerWorker_p0... | |
| [2023-02-24 08:13:02,314][796098] Loop learner_proc0_evt_loop terminating... | |
| [2023-02-24 08:13:11,768][795538] Loading existing experiment configuration from /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/config.json | |
| [2023-02-24 08:13:11,769][795538] Overriding arg 'num_workers' with value 1 passed from command line | |
| [2023-02-24 08:13:11,770][795538] Adding new argument 'no_render'=True that is not in the saved config file! | |
| [2023-02-24 08:13:11,770][795538] Adding new argument 'save_video'=True that is not in the saved config file! | |
| [2023-02-24 08:13:11,770][795538] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! | |
| [2023-02-24 08:13:11,771][795538] Adding new argument 'video_name'=None that is not in the saved config file! | |
| [2023-02-24 08:13:11,771][795538] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! | |
| [2023-02-24 08:13:11,772][795538] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! | |
| [2023-02-24 08:13:11,772][795538] Adding new argument 'push_to_hub'=True that is not in the saved config file! | |
| [2023-02-24 08:13:11,773][795538] Adding new argument 'hf_repository'='chqmatteo/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! | |
| [2023-02-24 08:13:11,773][795538] Adding new argument 'policy_index'=0 that is not in the saved config file! | |
| [2023-02-24 08:13:11,773][795538] Adding new argument 'eval_deterministic'=False that is not in the saved config file! | |
| [2023-02-24 08:13:11,774][795538] Adding new argument 'train_script'=None that is not in the saved config file! | |
| [2023-02-24 08:13:11,774][795538] Adding new argument 'enjoy_script'=None that is not in the saved config file! | |
| [2023-02-24 08:13:11,775][795538] Using frameskip 1 and render_action_repeat=4 for evaluation | |
| [2023-02-24 08:13:11,779][795538] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-02-24 08:13:11,780][795538] RunningMeanStd input shape: (1,) | |
| [2023-02-24 08:13:11,787][795538] ConvEncoder: input_channels=3 | |
| [2023-02-24 08:13:11,812][795538] Conv encoder output size: 512 | |
| [2023-02-24 08:13:11,813][795538] Policy head output size: 512 | |
| [2023-02-24 08:13:11,845][795538] Loading state from checkpoint /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000001361_5574656.pth... | |
| [2023-02-24 08:13:12,294][795538] Num frames 100... | |
| [2023-02-24 08:13:12,360][795538] Num frames 200... | |
| [2023-02-24 08:13:12,438][795538] Num frames 300... | |
| [2023-02-24 08:13:12,502][795538] Num frames 400... | |
| [2023-02-24 08:13:12,568][795538] Num frames 500... | |
| [2023-02-24 08:13:12,635][795538] Num frames 600... | |
| [2023-02-24 08:13:12,701][795538] Num frames 700... | |
| [2023-02-24 08:13:12,763][795538] Num frames 800... | |
| [2023-02-24 08:13:12,833][795538] Num frames 900... | |
| [2023-02-24 08:13:12,896][795538] Num frames 1000... | |
| [2023-02-24 08:13:12,971][795538] Num frames 1100... | |
| [2023-02-24 08:13:13,037][795538] Num frames 1200... | |
| [2023-02-24 08:13:13,102][795538] Avg episode rewards: #0: 29.160, true rewards: #0: 12.160 | |
| [2023-02-24 08:13:13,102][795538] Avg episode reward: 29.160, avg true_objective: 12.160 | |
| [2023-02-24 08:13:13,165][795538] Num frames 1300... | |
| [2023-02-24 08:13:13,228][795538] Num frames 1400... | |
| [2023-02-24 08:13:13,291][795538] Num frames 1500... | |
| [2023-02-24 08:13:13,362][795538] Num frames 1600... | |
| [2023-02-24 08:13:13,429][795538] Num frames 1700... | |
| [2023-02-24 08:13:13,494][795538] Num frames 1800... | |
| [2023-02-24 08:13:13,563][795538] Num frames 1900... | |
| [2023-02-24 08:13:13,627][795538] Num frames 2000... | |
| [2023-02-24 08:13:13,697][795538] Num frames 2100... | |
| [2023-02-24 08:13:13,770][795538] Num frames 2200... | |
| [2023-02-24 08:13:13,838][795538] Num frames 2300... | |
| [2023-02-24 08:13:13,919][795538] Num frames 2400... | |
| [2023-02-24 08:13:13,989][795538] Num frames 2500... | |
| [2023-02-24 08:13:14,053][795538] Num frames 2600... | |
| [2023-02-24 08:13:14,132][795538] Num frames 2700... | |
| [2023-02-24 08:13:14,205][795538] Num frames 2800... | |
| [2023-02-24 08:13:14,265][795538] Avg episode rewards: #0: 35.535, true rewards: #0: 14.035 | |
| [2023-02-24 08:13:14,266][795538] Avg episode reward: 35.535, avg true_objective: 14.035 | |
| [2023-02-24 08:13:14,332][795538] Num frames 2900... | |
| [2023-02-24 08:13:14,399][795538] Num frames 3000... | |
| [2023-02-24 08:13:14,464][795538] Num frames 3100... | |
| [2023-02-24 08:13:14,537][795538] Num frames 3200... | |
| [2023-02-24 08:13:14,627][795538] Num frames 3300... | |
| [2023-02-24 08:13:14,696][795538] Num frames 3400... | |
| [2023-02-24 08:13:14,807][795538] Avg episode rewards: #0: 28.950, true rewards: #0: 11.617 | |
| [2023-02-24 08:13:14,808][795538] Avg episode reward: 28.950, avg true_objective: 11.617 | |
| [2023-02-24 08:13:14,818][795538] Num frames 3500... | |
| [2023-02-24 08:13:14,882][795538] Num frames 3600... | |
| [2023-02-24 08:13:14,943][795538] Num frames 3700... | |
| [2023-02-24 08:13:15,002][795538] Num frames 3800... | |
| [2023-02-24 08:13:15,069][795538] Num frames 3900... | |
| [2023-02-24 08:13:15,133][795538] Num frames 4000... | |
| [2023-02-24 08:13:15,196][795538] Num frames 4100... | |
| [2023-02-24 08:13:15,259][795538] Num frames 4200... | |
| [2023-02-24 08:13:15,327][795538] Num frames 4300... | |
| [2023-02-24 08:13:15,403][795538] Num frames 4400... | |
| [2023-02-24 08:13:15,470][795538] Num frames 4500... | |
| [2023-02-24 08:13:15,534][795538] Num frames 4600... | |
| [2023-02-24 08:13:15,596][795538] Num frames 4700... | |
| [2023-02-24 08:13:15,660][795538] Num frames 4800... | |
| [2023-02-24 08:13:15,736][795538] Num frames 4900... | |
| [2023-02-24 08:13:15,804][795538] Num frames 5000... | |
| [2023-02-24 08:13:15,870][795538] Num frames 5100... | |
| [2023-02-24 08:13:15,944][795538] Num frames 5200... | |
| [2023-02-24 08:13:16,023][795538] Num frames 5300... | |
| [2023-02-24 08:13:16,098][795538] Num frames 5400... | |
| [2023-02-24 08:13:16,186][795538] Num frames 5500... | |
| [2023-02-24 08:13:16,306][795538] Avg episode rewards: #0: 35.962, true rewards: #0: 13.962 | |
| [2023-02-24 08:13:16,307][795538] Avg episode reward: 35.962, avg true_objective: 13.962 | |
| [2023-02-24 08:13:16,320][795538] Num frames 5600... | |
| [2023-02-24 08:13:16,393][795538] Num frames 5700... | |
| [2023-02-24 08:13:16,460][795538] Num frames 5800... | |
| [2023-02-24 08:13:16,524][795538] Num frames 5900... | |
| [2023-02-24 08:13:16,601][795538] Avg episode rewards: #0: 30.080, true rewards: #0: 11.880 | |
| [2023-02-24 08:13:16,602][795538] Avg episode reward: 30.080, avg true_objective: 11.880 | |
| [2023-02-24 08:13:16,640][795538] Num frames 6000... | |
| [2023-02-24 08:13:16,708][795538] Num frames 6100... | |
| [2023-02-24 08:13:16,794][795538] Num frames 6200... | |
| [2023-02-24 08:13:16,854][795538] Num frames 6300... | |
| [2023-02-24 08:13:16,920][795538] Num frames 6400... | |
| [2023-02-24 08:13:16,995][795538] Num frames 6500... | |
| [2023-02-24 08:13:17,060][795538] Num frames 6600... | |
| [2023-02-24 08:13:17,136][795538] Num frames 6700... | |
| [2023-02-24 08:13:17,202][795538] Num frames 6800... | |
| [2023-02-24 08:13:17,265][795538] Num frames 6900... | |
| [2023-02-24 08:13:17,334][795538] Num frames 7000... | |
| [2023-02-24 08:13:17,402][795538] Num frames 7100... | |
| [2023-02-24 08:13:17,467][795538] Num frames 7200... | |
| [2023-02-24 08:13:17,541][795538] Num frames 7300... | |
| [2023-02-24 08:13:17,614][795538] Num frames 7400... | |
| [2023-02-24 08:13:17,693][795538] Num frames 7500... | |
| [2023-02-24 08:13:17,761][795538] Num frames 7600... | |
| [2023-02-24 08:13:17,852][795538] Avg episode rewards: #0: 32.753, true rewards: #0: 12.753 | |
| [2023-02-24 08:13:17,853][795538] Avg episode reward: 32.753, avg true_objective: 12.753 | |
| [2023-02-24 08:13:17,891][795538] Num frames 7700... | |
| [2023-02-24 08:13:17,954][795538] Num frames 7800... | |
| [2023-02-24 08:13:18,026][795538] Num frames 7900... | |
| [2023-02-24 08:13:18,113][795538] Num frames 8000... | |
| [2023-02-24 08:13:18,178][795538] Num frames 8100... | |
| [2023-02-24 08:13:18,244][795538] Num frames 8200... | |
| [2023-02-24 08:13:18,314][795538] Num frames 8300... | |
| [2023-02-24 08:13:18,380][795538] Num frames 8400... | |
| [2023-02-24 08:13:18,452][795538] Num frames 8500... | |
| [2023-02-24 08:13:18,528][795538] Num frames 8600... | |
| [2023-02-24 08:13:18,601][795538] Num frames 8700... | |
| [2023-02-24 08:13:18,664][795538] Num frames 8800... | |
| [2023-02-24 08:13:18,732][795538] Num frames 8900... | |
| [2023-02-24 08:13:18,799][795538] Num frames 9000... | |
| [2023-02-24 08:13:18,876][795538] Num frames 9100... | |
| [2023-02-24 08:13:18,945][795538] Num frames 9200... | |
| [2023-02-24 08:13:19,017][795538] Num frames 9300... | |
| [2023-02-24 08:13:19,083][795538] Num frames 9400... | |
| [2023-02-24 08:13:19,162][795538] Num frames 9500... | |
| [2023-02-24 08:13:19,240][795538] Num frames 9600... | |
| [2023-02-24 08:13:19,308][795538] Num frames 9700... | |
| [2023-02-24 08:13:19,398][795538] Avg episode rewards: #0: 36.788, true rewards: #0: 13.931 | |
| [2023-02-24 08:13:19,398][795538] Avg episode reward: 36.788, avg true_objective: 13.931 | |
| [2023-02-24 08:13:19,431][795538] Num frames 9800... | |
| [2023-02-24 08:13:19,505][795538] Num frames 9900... | |
| [2023-02-24 08:13:19,570][795538] Num frames 10000... | |
| [2023-02-24 08:13:19,642][795538] Num frames 10100... | |
| [2023-02-24 08:13:19,704][795538] Num frames 10200... | |
| [2023-02-24 08:13:19,778][795538] Avg episode rewards: #0: 33.665, true rewards: #0: 12.790 | |
| [2023-02-24 08:13:19,779][795538] Avg episode reward: 33.665, avg true_objective: 12.790 | |
| [2023-02-24 08:13:19,831][795538] Num frames 10300... | |
| [2023-02-24 08:13:19,906][795538] Num frames 10400... | |
| [2023-02-24 08:13:20,011][795538] Num frames 10500... | |
| [2023-02-24 08:13:20,081][795538] Num frames 10600... | |
| [2023-02-24 08:13:20,144][795538] Num frames 10700... | |
| [2023-02-24 08:13:20,220][795538] Num frames 10800... | |
| [2023-02-24 08:13:20,322][795538] Avg episode rewards: #0: 31.302, true rewards: #0: 12.080 | |
| [2023-02-24 08:13:20,322][795538] Avg episode reward: 31.302, avg true_objective: 12.080 | |
| [2023-02-24 08:13:20,348][795538] Num frames 10900... | |
| [2023-02-24 08:13:20,415][795538] Num frames 11000... | |
| [2023-02-24 08:13:20,477][795538] Num frames 11100... | |
| [2023-02-24 08:13:20,537][795538] Num frames 11200... | |
| [2023-02-24 08:13:20,596][795538] Num frames 11300... | |
| [2023-02-24 08:13:20,656][795538] Num frames 11400... | |
| [2023-02-24 08:13:20,717][795538] Num frames 11500... | |
| [2023-02-24 08:13:20,774][795538] Num frames 11600... | |
| [2023-02-24 08:13:20,836][795538] Num frames 11700... | |
| [2023-02-24 08:13:20,897][795538] Num frames 11800... | |
| [2023-02-24 08:13:20,958][795538] Num frames 11900... | |
| [2023-02-24 08:13:21,028][795538] Num frames 12000... | |
| [2023-02-24 08:13:21,097][795538] Num frames 12100... | |
| [2023-02-24 08:13:21,160][795538] Num frames 12200... | |
| [2023-02-24 08:13:21,224][795538] Num frames 12300... | |
| [2023-02-24 08:13:21,292][795538] Num frames 12400... | |
| [2023-02-24 08:13:21,394][795538] Avg episode rewards: #0: 32.272, true rewards: #0: 12.472 | |
| [2023-02-24 08:13:21,395][795538] Avg episode reward: 32.272, avg true_objective: 12.472 | |
| [2023-02-24 08:13:27,245][795538] Replay video saved to /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/replay.mp4! | |