Training in progress, step 2000

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1be05096f1055ca0b3022a876a2827ad24ff96a673483fa8886e418ff25d1f07
+oid sha256:51dde3d5b9d7f85051df056de89e082563e3d018cb76f28d8224c82be9328a8c
 size 967102601

training.log

@@ -917,3 +917,106 @@ wandb: 🚀 View run at https://wandb.ai/danielizham/huggingface/runs/9zf5xxpu
 Adding files tracked by Git LFS: ['wandb/run-20230506_113337-ysywp688/run-ysywp688.wandb', 'wandb/run-20230507_103405-9zf5xxpu/run-9zf5xxpu.wandb']. This may take a bit of time if the files are large.
 {'eval_loss': 0.43405279517173767, 'eval_wer': 54.25600000000001, 'eval_runtime': 2248.2056, 'eval_samples_per_second': 4.644, 'eval_steps_per_second': 0.073, 'epoch': 6.0}
 05/07/2023 19:00:40 - WARNING - huggingface_hub.repository - Adding files tracked by Git LFS: ['wandb/run-20230506_113337-ysywp688/run-ysywp688.wandb', 'wandb/run-20230507_103405-9zf5xxpu/run-9zf5xxpu.wandb']. This may take a bit of time if the files are large.
+/home/local/QCRI/dizham/miniconda3/envs/whisper/lib/python3.9/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
+  warnings.warn('Was asked to gather along dimension 0, but all '
+
 20%|██        | 1001/5000 [8:27:07<791:34:47, 712.60s/it]
 20%|██        | 1002/5000 [8:27:35<563:11:28, 507.13s/it]
 20%|██        | 1003/5000 [8:28:04<403:46:05, 363.66s/it]
 20%|██        | 1004/5000 [8:28:32<291:54:08, 262.98s/it]
 20%|██        | 1005/5000 [8:28:59<213:24:18, 192.30s/it]
 20%|██        | 1006/5000 [8:29:28<158:59:55, 143.31s/it]
 20%|██        | 1007/5000 [8:29:56<120:19:03, 108.48s/it]
 20%|██        | 1008/5000 [8:30:23<93:12:59, 84.06s/it]  
 20%|██        | 1009/5000 [8:30:52<74:58:34, 67.63s/it]
 20%|██        | 1010/5000 [8:31:19<61:38:34, 55.62s/it]
 20%|██        | 1011/5000 [8:31:46<51:49:30, 46.77s/it]
 20%|██        | 1012/5000 [8:32:20<47:34:07, 42.94s/it]
 20%|██        | 1013/5000 [8:32:46<42:11:11, 38.09s/it]
 20%|██        | 1014/5000 [8:33:14<38:34:08, 34.83s/it]
 20%|██        | 1015/5000 [8:33:43<36:51:23, 33.30s/it]
 20%|██        | 1016/5000 [8:34:11<34:51:19, 31.50s/it]
 20%|██        | 1017/5000 [8:34:38<33:26:52, 30.23s/it]
 20%|██        | 1018/5000 [8:35:07<32:57:16, 29.79s/it]
 20%|██        | 1019/5000 [8:35:34<32:09:41, 29.08s/it]
 20%|██        | 1020/5000 [8:36:01<31:28:13, 28.47s/it]
 20%|██        | 1021/5000 [8:36:29<31:22:00, 28.38s/it]
 20%|██        | 1022/5000 [8:36:56<30:53:33, 27.96s/it]
 20%|██        | 1023/5000 [8:37:25<31:00:06, 28.06s/it]
 20%|██        | 1024/5000 [8:37:52<30:54:36, 27.99s/it]
 20%|██        | 1025/5000 [8:38:20<30:36:36, 27.72s/it]
                                                        
 20%|██        | 1025/5000 [8:38:20<30:36:36, 27.72s/it]
 21%|██        | 1026/5000 [8:38:52<32:15:09, 29.22s/it]
 21%|██        | 1027/5000 [8:39:20<31:39:01, 28.68s/it]
 21%|██        | 1028/5000 [8:39:49<31:48:02, 28.82s/it]
 21%|██        | 1029/5000 [8:40:17<31:30:25, 28.56s/it]
 21%|██        | 1030/5000 [8:40:44<31:02:35, 28.15s/it]
 21%|██        | 1031/5000 [8:41:12<30:58:22, 28.09s/it]
 21%|██        | 1032/5000 [8:41:40<31:03:55, 28.18s/it]
 21%|██        | 1033/5000 [8:42:08<30:48:01, 27.95s/it]
 21%|██        | 1034/5000 [8:42:37<31:18:07, 28.41s/it]
 21%|██        | 1035/5000 [8:43:04<30:48:47, 27.98s/it]
 21%|██        | 1036/5000 [8:43:31<30:34:22, 27.77s/it]
 21%|██        | 1037/5000 [8:44:01<31:08:55, 28.30s/it]
 21%|██        | 1038/5000 [8:44:28<30:52:59, 28.06s/it]
 21%|██        | 1039/5000 [8:44:55<30:22:36, 27.61s/it]
 21%|██        | 1040/5000 [8:45:24<30:47:47, 28.00s/it]
 21%|██        | 1041/5000 [8:45:51<30:35:37, 27.82s/it]
 21%|██        | 1042/5000 [8:46:19<30:27:20, 27.70s/it]
 21%|██        | 1043/5000 [8:46:48<30:51:34, 28.08s/it]
 21%|██        | 1044/5000 [8:47:15<30:37:28, 27.87s/it]
 21%|██        | 1045/5000 [8:47:43<30:27:59, 27.73s/it]
 21%|██        | 1046/5000 [8:48:13<31:16:19, 28.47s/it]
 21%|██        | 1047/5000 [8:48:40<30:57:09, 28.19s/it]
 21%|██        | 1048/5000 [8:49:08<30:57:21, 28.20s/it]
 21%|██        | 1049/5000 [8:49:36<30:47:50, 28.06s/it]
 21%|██        | 1050/5000 [8:50:03<30:27:46, 27.76s/it]
                                                        
 21%|██        | 1050/5000 [8:50:03<30:27:46, 27.76s/it]
 21%|██        | 1051/5000 [8:50:31<30:35:30, 27.89s/it]
 21%|██        | 1052/5000 [8:50:59<30:23:13, 27.71s/it]
 21%|██        | 1053/5000 [8:51:26<30:20:08, 27.67s/it]
 21%|██        | 1054/5000 [8:51:55<30:40:01, 27.98s/it]
 21%|██        | 1055/5000 [8:52:22<30:26:27, 27.78s/it]
 21%|██        | 1056/5000 [8:52:51<30:35:40, 27.93s/it]
 21%|██        | 1057/5000 [8:53:19<30:49:56, 28.15s/it]
 21%|██        | 1058/5000 [8:53:47<30:32:42, 27.90s/it]
 21%|██        | 1059/5000 [8:54:15<30:49:41, 28.16s/it]
 21%|██        | 1060/5000 [8:54:44<30:49:17, 28.16s/it]
 21%|██        | 1061/5000 [8:55:11<30:35:19, 27.96s/it]
 21%|██        | 1062/5000 [8:55:39<30:31:43, 27.91s/it]
 21%|██▏       | 1063/5000 [8:56:07<30:35:45, 27.98s/it]
 21%|██▏       | 1064/5000 [8:56:34<30:18:02, 27.71s/it]
 21%|██▏       | 1065/5000 [8:57:02<30:22:09, 27.78s/it]
 21%|██▏       | 1066/5000 [8:57:30<30:19:09, 27.75s/it]
 21%|██▏       | 1067/5000 [8:57:58<30:32:30, 27.96s/it]
 21%|██▏       | 1068/5000 [8:58:26<30:22:01, 27.80s/it]
 21%|██▏       | 1069/5000 [8:58:53<30:23:43, 27.84s/it]
 21%|██▏       | 1070/5000 [8:59:22<30:30:16, 27.94s/it]
 21%|██▏       | 1071/5000 [8:59:49<30:15:17, 27.72s/it]
 21%|██▏       | 1072/5000 [9:00:17<30:25:31, 27.88s/it]
 21%|██▏       | 1073/5000 [9:00:45<30:29:45, 27.96s/it]
 21%|██▏       | 1074/5000 [9:01:12<30:10:29, 27.67s/it]
 22%|██▏       | 1075/5000 [9:01:40<30:19:39, 27.82s/it]
                                                        
 22%|██▏       | 1075/5000 [9:01:40<30:19:39, 27.82s/it]
 22%|██▏       | 1076/5000 [9:02:09<30:31:41, 28.01s/it]
 22%|██▏       | 1077/5000 [9:02:36<30:21:10, 27.85s/it]
 22%|██▏       | 1078/5000 [9:03:10<32:09:28, 29.52s/it]
 22%|██▏       | 1079/5000 [9:03:38<31:35:39, 29.01s/it]
 22%|██▏       | 1080/5000 [9:04:05<31:02:48, 28.51s/it]
 22%|██▏       | 1081/5000 [9:04:34<31:04:38, 28.55s/it]
 22%|██▏       | 1082/5000 [9:05:01<30:49:32, 28.32s/it]
 22%|██▏       | 1083/5000 [9:05:29<30:27:31, 27.99s/it]
 22%|██▏       | 1084/5000 [9:05:57<30:35:11, 28.12s/it]
 22%|██▏       | 1085/5000 [9:06:24<30:16:06, 27.83s/it]
 22%|██▏       | 1086/5000 [9:06:51<30:05:13, 27.67s/it]
 22%|██▏       | 1087/5000 [9:07:19<30:12:16, 27.79s/it]
 22%|██▏       | 1088/5000 [9:07:47<29:59:56, 27.61s/it]
 22%|██▏       | 1089/5000 [9:08:14<29:53:36, 27.52s/it]
 22%|██▏       | 1090/5000 [9:08:42<30:12:33, 27.81s/it]
 22%|██▏       | 1091/5000 [9:09:11<30:22:11, 27.97s/it]
 22%|██▏       | 1092/5000 [9:09:38<30:04:13, 27.70s/it]
 22%|██▏       | 1093/5000 [9:10:07<30:23:16, 28.00s/it]
 22%|██▏       | 1094/5000 [9:10:35<30:28:50, 28.09s/it]
 22%|██▏       | 1095/5000 [9:11:02<30:13:00, 27.86s/it]
 22%|██▏       | 1096/5000 [9:11:30<30:06:49, 27.77s/it]
 22%|██▏       | 1097/5000 [9:11:57<29:52:09, 27.55s/it]
 22%|██▏       | 1098/5000 [9:12:24<29:39:33, 27.36s/it]
 22%|██▏       | 1099/5000 [9:12:51<29:28:49, 27.21s/it]
 22%|██▏       | 1100/5000 [9:13:19<29:59:10, 27.68s/it]
                                                        
 22%|██▏       | 1100/5000 [9:13:19<29:59:10, 27.68s/it]
 22%|██▏       | 1101/5000 [9:13:47<29:51:50, 27.57s/it]
 22%|██▏       | 1102/5000 [9:14:14<29:47:06, 27.51s/it]
 22%|██▏       | 1103/5000 [9:14:44<30:29:52, 28.17s/it]
 22%|██▏       | 1104/5000 [9:15:11<30:10:21, 27.88s/it]
 22%|██▏       | 1105/5000 [9:15:38<29:59:50, 27.73s/it]
 22%|██▏       | 1106/5000 [9:16:07<30:17:25, 28.00s/it]
 22%|██▏       | 1107/5000 [9:16:34<29:53:47, 27.65s/it]
 22%|██▏       | 1108/5000 [9:17:01<29:46:04, 27.53s/it]
 22%|██▏       | 1109/5000 [9:17:30<30:16:16, 28.01s/it]
 22%|██▏       | 1110/5000 [9:17:56<29:32:50, 27.34s/it]
 22%|██▏       | 1111/5000 [9:18:22<29:10:58, 27.01s/it]
 22%|██▏       | 1112/5000 [9:18:49<29:11:25, 27.03s/it]
 22%|██▏       | 1113/5000 [9:19:21<30:44:52, 28.48s/it]
 22%|██▏       | 1114/5000 [9:19:48<30:21:43, 28.13s/it]
 22%|██▏       | 1115/5000 [9:20:16<30:03:12, 27.85s/it]
 22%|██▏       | 1116/5000 [9:20:44<30:16:39, 28.06s/it]
 22%|██▏       | 1117/5000 [9:21:11<29:59:08, 27.80s/it]
 22%|██▏       | 1118/5000 [9:21:39<29:47:14, 27.62s/it]
 22%|██▏       | 1119/5000 [9:22:08<30:25:13, 28.22s/it]
 22%|██▏       | 1120/5000 [9:22:35<30:04:44, 27.91s/it]
 22%|██▏       | 1121/5000 [9:23:03<29:55:42, 27.78s/it]
 22%|██▏       | 1122/5000 [9:23:31<30:02:09, 27.88s/it]
 22%|██▏       | 1123/5000 [9:23:59<29:54:51, 27.78s/it]
 22%|██▏       | 1124/5000 [9:24:26<29:46:03, 27.65s/it]
 22%|██▎       | 1125/5000 [9:24:55<30:10:55, 28.04s/it]
                                                        
 22%|██▎       | 1125/5000 [9:24:55<30:10:55, 28.04s/it]
 23%|██▎       | 1126/5000 [9:25:22<29:51:53, 27.75s/it]
 23%|██▎       | 1127/5000 [9:25:49<29:45:21, 27.66s/it]
 23%|██▎       | 1128/5000 [9:26:20<30:44:37, 28.58s/it]
 23%|██▎       | 1129/5000 [9:26:47<30:10:15, 28.06s/it]
 23%|██▎       | 1130/5000 [9:27:14<29:55:46, 27.84s/it]
 23%|██▎       | 1131/5000 [9:27:44<30:27:45, 28.34s/it]
 23%|██▎       | 1132/5000 [9:28:11<30:09:16, 28.07s/it]
 23%|██▎       | 1133/5000 [9:28:39<29:54:37, 27.85s/it]
 23%|██▎       | 1134/5000 [9:29:07<30:12:17, 28.13s/it]
 23%|██▎       | 1135/5000 [9:29:35<29:55:13, 27.87s/it]
 23%|██▎       | 1136/5000 [9:30:02<29:43:08, 27.69s/it]
 23%|██▎       | 1137/5000 [9:30:31<30:11:03, 28.13s/it]
 23%|██▎       | 1138/5000 [9:30:58<29:51:31, 27.83s/it]
 23%|██▎       | 1139/5000 [9:31:25<29:39:25, 27.65s/it]
 23%|██▎       | 1140/5000 [9:31:55<30:11:51, 28.16s/it]
 23%|██▎       | 1141/5000 [9:32:08<25:33:16, 23.84s/it]
 23%|██▎       | 1142/5000 [9:32:19<21:24:24, 19.98s/it]
 23%|██▎       | 1143/5000 [9:32:30<18:27:07, 17.22s/it]
 23%|██▎       | 1144/5000 [9:32:41<16:23:31, 15.30s/it]{'loss': 0.0512, 'learning_rate': 8.83777777777778e-06, 'epoch': 6.01}
+{'loss': 0.048, 'learning_rate': 8.782222222222223e-06, 'epoch': 6.01}
+{'loss': 0.034, 'learning_rate': 8.726666666666667e-06, 'epoch': 6.02}
+{'loss': 0.0415, 'learning_rate': 8.671111111111113e-06, 'epoch': 6.02}
+{'loss': 0.0402, 'learning_rate': 8.615555555555555e-06, 'epoch': 6.03}
+
+
+
+
+
+
+
+
+
 23%|██▎       | 1145/5000 [9:34:08<39:27:24, 36.85s/it]
 23%|██▎       | 1146/5000 [9:34:36<36:36:12, 34.19s/it]
 23%|██▎       | 1147/5000 [9:35:04<34:35:24, 32.32s/it]
 23%|██▎       | 1148/5000 [9:35:33<33:24:02, 31.22s/it]
 23%|██▎       | 1149/5000 [9:36:00<32:12:01, 30.10s/it]
 23%|██▎       | 1150/5000 [9:36:28<31:22:23, 29.34s/it]
                                                        
 23%|██▎       | 1150/5000 [9:36:28<31:22:23, 29.34s/it]
 23%|██▎       | 1151/5000 [9:36:56<30:57:29, 28.96s/it]
 23%|██▎       | 1152/5000 [9:37:25<31:06:31, 29.10s/it]
 23%|██▎       | 1153/5000 [9:37:54<30:52:06, 28.89s/it]
 23%|██▎       | 1154/5000 [9:38:21<30:24:52, 28.47s/it]
 23%|██▎       | 1155/5000 [9:38:49<30:20:14, 28.40s/it]
 23%|██▎       | 1156/5000 [9:39:18<30:20:51, 28.42s/it]
 23%|██▎       | 1157/5000 [9:39:45<30:01:50, 28.13s/it]
 23%|██▎       | 1158/5000 [9:40:17<31:17:11, 29.32s/it]
 23%|██▎       | 1159/5000 [9:40:46<31:04:33, 29.13s/it]
 23%|██▎       | 1160/5000 [9:41:14<30:47:06, 28.86s/it]
 23%|██▎       | 1161/5000 [9:41:42<30:31:23, 28.62s/it]
 23%|██▎       | 1162/5000 [9:42:09<30:00:08, 28.14s/it]
 23%|██▎       | 1163/5000 [9:42:38<30:07:34, 28.27s/it]
 23%|██▎       | 1164/5000 [9:43:07<30:22:38, 28.51s/it]
 23%|██▎       | 1165/5000 [9:43:34<29:54:41, 28.08s/it]
 23%|██▎       | 1166/5000 [9:44:03<29:59:17, 28.16s/it]
 23%|██▎       | 1167/5000 [9:44:30<29:46:29, 27.96s/it]
 23%|██▎       | 1168/5000 [9:44:57<29:33:03, 27.76s/it]
 23%|██▎       | 1169/5000 [9:45:26<29:42:04, 27.91s/it]
 23%|██▎       | 1170/5000 [9:45:54<29:43:32, 27.94s/it]
 23%|██▎       | 1171/5000 [9:46:21<29:24:42, 27.65s/it]
 23%|██▎       | 1172/5000 [9:46:48<29:28:19, 27.72s/it]
 23%|██▎       | 1173/5000 [9:47:16<29:26:16, 27.69s/it]
 23%|██▎       | 1174/5000 [9:47:44<29:21:08, 27.62s/it]
 24%|██▎       | 1175/5000 [9:48:12<29:43:46, 27.98s/it]
                                                        
 24%|██▎       | 1175/5000 [9:48:12<29:43:46, 27.98s/it]
 24%|██▎       | 1176/5000 [9:48:40<29:35:28, 27.86s/it]
 24%|██▎       | 1177/5000 [9:49:07<29:24:07, 27.69s/it]
 24%|██▎       | 1178/5000 [9:49:37<30:00:47, 28.27s/it]
 24%|██▎       | 1179/5000 [9:50:04<29:40:06, 27.95s/it]
 24%|██▎       | 1180/5000 [9:50:31<29:28:19, 27.77s/it]
 24%|██▎       | 1181/5000 [9:51:00<29:45:25, 28.05s/it]
 24%|██▎       | 1182/5000 [9:51:27<29:27:17, 27.77s/it]
 24%|██▎       | 1183/5000 [9:51:55<29:18:55, 27.65s/it]
 24%|██▎       | 1184/5000 [9:52:22<29:17:14, 27.63s/it]
 24%|██▎       | 1185/5000 [9:52:51<29:35:04, 27.92s/it]
 24%|██▎       | 1186/5000 [9:53:18<29:12:33, 27.57s/it]
 24%|██▎       | 1187/5000 [9:53:46<29:28:17, 27.83s/it]
 24%|██▍       | 1188/5000 [9:54:14<29:28:42, 27.84s/it]
 24%|██▍       | 1189/5000 [9:54:41<29:19:26, 27.70s/it]
 24%|██▍       | 1190/5000 [9:55:09<29:21:17, 27.74s/it]
 24%|██▍       | 1191/5000 [9:55:36<29:16:01, 27.66s/it]
 24%|██▍       | 1192/5000 [9:56:05<29:27:45, 27.85s/it]
 24%|██▍       | 1193/5000 [9:56:32<29:21:57, 27.77s/it]
 24%|██▍       | 1194/5000 [9:57:00<29:12:55, 27.63s/it]
 24%|██▍       | 1195/5000 [9:57:28<29:26:52, 27.86s/it]
 24%|██▍       | 1196/5000 [9:57:55<29:17:25, 27.72s/it]
 24%|██▍       | 1197/5000 [9:58:28<30:39:41, 29.02s/it]
 24%|██▍       | 1198/5000 [9:58:55<30:08:15, 28.54s/it]
 24%|██▍       | 1199/5000 [9:59:22<29:47:08, 28.21s/it]
 24%|██▍       | 1200/5000 [9:59:51<29:47:38, 28.23s/it]
                                                        
 24%|██▍       | 1200/5000 [9:59:51<29:47:38, 28.23s/it]
 24%|██▍       | 1201/5000 [10:00:19<29:50:16, 28.27s/it]
 24%|██▍       | 1202/5000 [10:00:46<29:28:02, 27.93s/it]
 24%|██▍       | 1203/5000 [10:01:16<30:07:10, 28.56s/it]
 24%|██▍       | 1204/5000 [10:01:44<29:44:04, 28.20s/it]
 24%|██▍       | 1205/5000 [10:02:10<29:11:59, 27.70s/it]
 24%|██▍       | 1206/5000 [10:02:39<29:31:12, 28.01s/it]
 24%|██▍       | 1207/5000 [10:03:06<29:21:01, 27.86s/it]
 24%|██▍       | 1208/5000 [10:03:34<29:12:16, 27.73s/it]
 24%|██▍       | 1209/5000 [10:04:02<29:14:38, 27.77s/it]
 24%|██▍       | 1210/5000 [10:04:29<29:15:48, 27.80s/it]
 24%|██▍       | 1211/5000 [10:04:57<29:09:36, 27.71s/it]
 24%|██▍       | 1212/5000 [10:05:25<29:14:42, 27.79s/it]
 24%|██▍       | 1213/5000 [10:05:54<29:29:44, 28.04s/it]
 24%|██▍       | 1214/5000 [10:06:21<29:14:43, 27.81s/it]
 24%|██▍       | 1215/5000 [10:06:49<29:27:37, 28.02s/it]
 24%|██▍       | 1216/5000 [10:07:17<29:20:27, 27.91s/it]
 24%|██▍       | 1217/5000 [10:07:44<29:05:54, 27.69s/it]
 24%|██▍       | 1218/5000 [10:08:13<29:28:06, 28.05s/it]
 24%|██▍       | 1219/5000 [10:08:42<29:37:11, 28.20s/it]
 24%|██▍       | 1220/5000 [10:09:10<29:38:02, 28.22s/it]
 24%|██▍       | 1221/5000 [10:09:37<29:23:29, 28.00s/it]
 24%|██▍       | 1222/5000 [10:10:05<29:17:45, 27.92s/it]
 24%|██▍       | 1223/5000 [10:10:34<29:29:06, 28.10s/it]
 24%|██▍       | 1224/5000 [10:11:01<29:19:05, 27.95s/it]
 24%|██▍       | 1225/5000 [10:11:33<30:38:14, 29.22s/it]
                                                         
 24%|██▍       | 1225/5000 [10:11:33<30:38:14, 29.22s/it]
 25%|██▍       | 1226/5000 [10:12:06<31:41:12, 30.23s/it]
 25%|██▍       | 1227/5000 [10:12:33<30:48:50, 29.40s/it]
 25%|██▍       | 1228/5000 [10:13:02<30:26:55, 29.06s/it]
 25%|██▍       | 1229/5000 [10:13:29<30:01:58, 28.67s/it]
 25%|██▍       | 1230/5000 [10:13:57<29:40:08, 28.33s/it]
 25%|██▍       | 1231/5000 [10:14:26<29:43:02, 28.38s/it]
 25%|██▍       | 1232/5000 [10:14:54<29:44:59, 28.42s/it]
 25%|██▍       | 1233/5000 [10:15:22<29:28:20, 28.17s/it]
 25%|██▍       | 1234/5000 [10:15:50<29:33:45, 28.26s/it]
 25%|██▍       | 1235/5000 [10:16:18<29:25:23, 28.13s/it]
 25%|██▍       | 1236/5000 [10:16:45<29:09:52, 27.89s/it]
 25%|██▍       | 1237/5000 [10:17:14<29:28:07, 28.19s/it]
 25%|██▍       | 1238/5000 [10:17:43<29:34:26, 28.30s/it]
 25%|██▍       | 1239/5000 [10:18:10<29:20:32, 28.09s/it]
 25%|██▍       | 1240/5000 [10:18:38<29:15:22, 28.01s/it]
 25%|██▍       | 1241/5000 [10:19:07<29:26:21, 28.19s/it]
 25%|██▍       | 1242/5000 [10:19:34<29:11:01, 27.96s/it]
 25%|██▍       | 1243/5000 [10:20:02<29:14:14, 28.02s/it]
 25%|██▍       | 1244/5000 [10:20:32<29:41:58, 28.47s/it]
 25%|██▍       | 1245/5000 [10:21:00<29:39:22, 28.43s/it]
 25%|██▍       | 1246/5000 [10:21:28<29:18:40, 28.11s/it]
 25%|██▍       | 1247/5000 [10:21:56<29:20:01, 28.14s/it]
 25%|██▍       | 1248/5000 [10:22:24<29:23:37, 28.20s/it]
 25%|██▍       | 1249/5000 [10:22:52<29:08:23, 27.97s/it]
 25%|██▌       | 1250/5000 [10:23:19<28:54:41, 27.75s/it]
                                                         
 25%|██▌       | 1250/5000 [10:23:19<28:54:41, 27.75s/it]
 25%|██▌       | 1251/5000 [10:23:47<29:01:29, 27.87s/it]
 25%|██▌       | 1252/5000 [10:24:14<28:50:15, 27.70s/it]
 25%|██▌       | 1253/5000 [10:24:41<28:39:31, 27.53s/it]
 25%|██▌       | 1254/5000 [10:25:10<29:06:11, 27.97s/it]
 25%|██▌       | 1255/5000 [10:25:38<28:52:51, 27.76s/it]
 25%|██▌       | 1256/5000 [10:26:05<28:41:32, 27.59s/it]
 25%|██▌       | 1257/5000 [10:26:34<29:16:45, 28.16s/it]
 25%|██▌       | 1258/5000 [10:27:02<29:00:11, 27.90s/it]
 25%|██▌       | 1259/5000 [10:27:34<30:29:19, 29.34s/it]
 25%|██▌       | 1260/5000 [10:28:03<30:23:14, 29.25s/it]
 25%|██▌       | 1261/5000 [10:28:32<30:06:56, 29.00s/it]
 25%|██▌       | 1262/5000 [10:28:59<29:33:12, 28.46s/it]
 25%|██▌       | 1263/5000 [10:29:28<29:37:23, 28.54s/it]
 25%|██▌       | 1264/5000 [10:29:56<29:31:45, 28.45s/it]
 25%|██▌       | 1265/5000 [10:30:23<29:09:47, 28.11s/it]
 25%|██▌       | 1266/5000 [10:30:53<29:31:51, 28.47s/it]
 25%|██▌       | 1267/5000 [10:31:20<29:17:45, 28.25s/it]
 25%|██▌       | 1268/5000 [10:31:48<29:01:45, 28.00s/it]
 25%|██▌       | 1269/5000 [10:32:19<29:57:39, 28.91s/it]
 25%|██▌       | 1270/5000 [10:32:47<29:37:03, 28.59s/it]
 25%|██▌       | 1271/5000 [10:33:14<29:16:45, 28.27s/it]
 25%|██▌       | 1272/5000 [10:33:41<28:57:17, 27.96s/it]
 25%|██▌       | 1273/5000 [10:34:11<29:25:01, 28.41s/it]
 25%|██▌       | 1274/5000 [10:34:38<29:04:30, 28.09s/it]
 26%|██▌       | 1275/5000 [10:35:05<28:48:22, 27.84s/it]
                                                         
 26%|██▌       | 1275/5000 [10:35:05<28:48:22, 27.84s/it]
 26%|██▌       | 1276/5000 [10:35:35<29:21:57, 28.39s/it]
 26%|██▌       | 1277/5000 [10:36:02<29:00:15, 28.05s/it]
 26%|██▌       | 1278/5000 [10:36:29<28:42:16, 27.76s/it]
 26%|██▌       | 1279/5000 [10:37:01<29:54:01, 28.93s/it]
 26%|██▌       | 1280/5000 [10:37:28<29:21:26, 28.41s/it]
 26%|██▌       | 1281/5000 [10:37:56<28:59:06, 28.06s/it]
 26%|██▌       | 1282/5000 [10:38:24<28:58:21, 28.05s/it]
 26%|██▌       | 1283/5000 [10:38:52<29:02:29, 28.13s/it]
 26%|██▌       | 1284/5000 [10:39:19<28:46:09, 27.87s/it]
 26%|██▌       | 1285/5000 [10:39:46<28:25:42, 27.55s/it]
 26%|██▌       | 1286/5000 [10:40:14<28:31:23, 27.65s/it]
 26%|██▌       | 1287/5000 [10:40:41<28:24:06, 27.54s/it]
 26%|██▌       | 1288/5000 [10:41:11<28:59:54, 28.12s/it]
 26%|██▌       | 1289/5000 [10:41:38<28:53:49, 28.03s/it]
 26%|██▌       | 1290/5000 [10:42:06<28:38:34, 27.79s/it]
 26%|██▌       | 1291/5000 [10:42:34<28:48:10, 27.96s/it]
 26%|██▌       | 1292/5000 [10:43:02<28:48:21, 27.97s/it]
 26%|██▌       | 1293/5000 [10:43:29<28:36:59, 27.79s/it]
 26%|██▌       | 1294/5000 [10:43:58<28:51:53, 28.04s/it]
 26%|██▌       | 1295/5000 [10:44:25<28:38:48, 27.84s/it]
 26%|██▌       | 1296/5000 [10:44:53<28:29:34, 27.69s/it]
 26%|██▌       | 1297/5000 [10:45:22<28:57:55, 28.16s/it]
 26%|██▌       | 1298/5000 [10:45:49<28:43:09, 27.93s/it]
 26%|██▌       | 1299/5000 [10:46:17<28:38:13, 27.86s/it]
 26%|██▌       | 1300/5000 [10:46:45<28:46:40, 28.00s/it]
                                                         
 26%|██▌       | 1300/5000 [10:46:45<28:46:40, 28.00s/it]
 26%|██▌       | 1301/5000 [10:47:12<28:29:08, 27.72s/it]
 26%|██▌       | 1302/5000 [10:47:40<28:18:07, 27.55s/it]
 26%|██▌       | 1303/5000 [10:48:09<28:49:19, 28.07s/it]
 26%|██▌       | 1304/5000 [10:48:31<27:00:51, 26.31s/it]
 26%|██▌       | 1305/5000 [10:48:42<22:15:13, 21.68s/it]
 26%|██▌       | 1306/5000 [10:48:53<18:52:30, 18.39s/it]
 26%|██▌       | 1307/5000 [10:49:04<16:33:11, 16.14s/it]
 26%|██▌       | 1308/5000 [10:49:12<14:02:21, 13.69s/it]{'loss': 0.0321, 'learning_rate': 8.560000000000001e-06, 'epoch': 7.0}
+{'loss': 0.0312, 'learning_rate': 8.504444444444445e-06, 'epoch': 7.01}
+{'loss': 0.0309, 'learning_rate': 8.448888888888889e-06, 'epoch': 7.01}
+{'loss': 0.0235, 'learning_rate': 8.393333333333335e-06, 'epoch': 7.02}
+{'loss': 0.0225, 'learning_rate': 8.337777777777777e-06, 'epoch': 7.02}
+{'loss': 0.0255, 'learning_rate': 8.282222222222223e-06, 'epoch': 7.03}
+{'loss': 0.0221, 'learning_rate': 8.226666666666667e-06, 'epoch': 7.03}
+
+
+
+
+
+
+
+
+
 26%|██▌       | 1309/5000 [10:50:51<40:30:13, 39.51s/it]
 26%|██▌       | 1310/5000 [10:51:19<36:51:28, 35.96s/it]
 26%|██▌       | 1311/5000 [10:51:46<34:15:18, 33.43s/it]
 26%|██▌       | 1312/5000 [10:52:15<32:48:56, 32.03s/it]
 26%|██▋       | 1313/5000 [10:52:44<31:42:44, 30.96s/it]
 26%|██▋       | 1314/5000 [10:53:12<30:44:16, 30.02s/it]
 26%|██▋       | 1315/5000 [10:53:38<29:47:00, 29.10s/it]
 26%|██▋       | 1316/5000 [10:54:07<29:30:34, 28.84s/it]
 26%|██▋       | 1317/5000 [10:54:35<29:23:11, 28.72s/it]
 26%|██▋       | 1318/5000 [10:55:02<28:47:46, 28.15s/it]
 26%|██▋       | 1319/5000 [10:55:29<28:34:03, 27.94s/it]
 26%|██▋       | 1320/5000 [10:55:57<28:19:25, 27.71s/it]
 26%|██▋       | 1321/5000 [10:56:24<28:13:39, 27.62s/it]
 26%|██▋       | 1322/5000 [10:56:52<28:27:31, 27.86s/it]
 26%|██▋       | 1323/5000 [10:57:20<28:20:03, 27.74s/it]
 26%|██▋       | 1324/5000 [10:57:48<28:34:52, 27.99s/it]
 26%|██▋       | 1325/5000 [10:58:18<29:02:53, 28.46s/it]
                                                         
 26%|██▋       | 1325/5000 [10:58:18<29:02:53, 28.46s/it]
 27%|██▋       | 1326/5000 [10:58:45<28:42:25, 28.13s/it]
 27%|██▋       | 1327/5000 [10:59:14<28:55:53, 28.36s/it]
 27%|██▋       | 1328/5000 [10:59:42<28:36:24, 28.05s/it]
 27%|██▋       | 1329/5000 [11:00:09<28:20:46, 27.80s/it]
 27%|██▋       | 1330/5000 [11:00:37<28:19:44, 27.79s/it]
 27%|██▋       | 1331/5000 [11:01:04<28:12:05, 27.67s/it]
 27%|██▋       | 1332/5000 [11:01:31<27:58:19, 27.45s/it]
 27%|██▋       | 1333/5000 [11:02:00<28:22:16, 27.85s/it]
 27%|██▋       | 1334/5000 [11:02:27<28:12:17, 27.70s/it]
 27%|██▋       | 1335/5000 [11:02:54<28:02:03, 27.54s/it]
 27%|██▋       | 1336/5000 [11:03:23<28:19:08, 27.82s/it]
 27%|██▋       | 1337/5000 [11:03:50<28:14:55, 27.76s/it]
 27%|██▋       | 1338/5000 [11:04:18<28:08:07, 27.66s/it]
 27%|██▋       | 1339/5000 [11:04:46<28:27:51, 27.99s/it]
 27%|██▋       | 1340/5000 [11:05:14<28:17:34, 27.83s/it]
 27%|██▋       | 1341/5000 [11:05:41<28:04:00, 27.61s/it]
 27%|██▋       | 1342/5000 [11:06:11<28:41:41, 28.24s/it]
 27%|██▋       | 1343/5000 [11:06:37<28:07:48, 27.69s/it]
 27%|██▋       | 1344/5000 [11:07:04<27:55:34, 27.50s/it]
 27%|██▋       | 1345/5000 [11:07:33<28:16:45, 27.85s/it]
 27%|██▋       | 1346/5000 [11:08:00<28:10:19, 27.76s/it]
 27%|██▋       | 1347/5000 [11:08:28<27:58:30, 27.57s/it]
 27%|██▋       | 1348/5000 [11:08:57<28:30:17, 28.10s/it]
 27%|██▋       | 1349/5000 [11:09:24<28:12:24, 27.81s/it]
 27%|██▋       | 1350/5000 [11:09:55<29:06:48, 28.71s/it]
                                                         
 27%|██▋       | 1350/5000 [11:09:55<29:06:48, 28.71s/it]
 27%|██▋       | 1351/5000 [11:10:24<29:08:35, 28.75s/it]
 27%|██▋       | 1352/5000 [11:10:51<28:39:20, 28.28s/it]
 27%|██▋       | 1353/5000 [11:11:24<29:59:24, 29.60s/it]
 27%|██▋       | 1354/5000 [11:11:51<29:19:51, 28.96s/it]
 27%|██▋       | 1355/5000 [11:12:18<28:48:42, 28.46s/it]
 27%|██▋       | 1356/5000 [11:12:46<28:43:07, 28.37s/it]
 27%|██▋       | 1357/5000 [11:13:14<28:23:55, 28.06s/it]
 27%|██▋       | 1358/5000 [11:13:41<28:15:04, 27.93s/it]
 27%|██▋       | 1359/5000 [11:14:09<28:17:31, 27.97s/it]
 27%|██▋       | 1360/5000 [11:14:37<28:05:50, 27.79s/it]
 27%|██▋       | 1361/5000 [11:15:05<28:16:14, 27.97s/it]
 27%|██▋       | 1362/5000 [11:15:33<28:04:12, 27.78s/it]
 27%|██▋       | 1363/5000 [11:16:00<27:57:20, 27.67s/it]
 27%|██▋       | 1364/5000 [11:16:29<28:24:49, 28.13s/it]
 27%|██▋       | 1365/5000 [11:16:56<28:03:22, 27.79s/it]
 27%|██▋       | 1366/5000 [11:17:23<27:53:22, 27.63s/it]
 27%|██▋       | 1367/5000 [11:17:52<28:03:10, 27.80s/it]
 27%|██▋       | 1368/5000 [11:18:18<27:42:37, 27.47s/it]
 27%|██▋       | 1369/5000 [11:18:45<27:30:31, 27.27s/it]
 27%|██▋       | 1370/5000 [11:19:15<28:13:04, 27.98s/it]
 27%|██▋       | 1371/5000 [11:19:42<27:57:43, 27.74s/it]
 27%|██▋       | 1372/5000 [11:20:08<27:32:56, 27.34s/it]
 27%|██▋       | 1373/5000 [11:20:38<28:06:40, 27.90s/it]
 27%|██▋       | 1374/5000 [11:21:05<27:52:56, 27.68s/it]
 28%|██▊       | 1375/5000 [11:21:33<28:04:21, 27.88s/it]
                                                         
 28%|██▊       | 1375/5000 [11:21:33<28:04:21, 27.88s/it]
 28%|██▊       | 1376/5000 [11:22:01<28:08:49, 27.96s/it]
 28%|██▊       | 1377/5000 [11:22:29<27:59:07, 27.81s/it]
 28%|██▊       | 1378/5000 [11:22:57<28:08:09, 27.96s/it]
 28%|██▊       | 1379/5000 [11:23:24<27:54:55, 27.75s/it]
 28%|██▊       | 1380/5000 [11:23:52<27:58:23, 27.82s/it]
 28%|██▊       | 1381/5000 [11:24:20<28:04:00, 27.92s/it]
 28%|██▊       | 1382/5000 [11:24:47<27:48:29, 27.67s/it]
 28%|██▊       | 1383/5000 [11:25:15<27:44:58, 27.62s/it]
 28%|██▊       | 1384/5000 [11:25:42<27:26:25, 27.32s/it]
 28%|██▊       | 1385/5000 [11:26:09<27:23:13, 27.27s/it]
 28%|██▊       | 1386/5000 [11:26:37<27:44:26, 27.63s/it]
 28%|██▊       | 1387/5000 [11:27:05<27:50:53, 27.75s/it]
 28%|██▊       | 1388/5000 [11:27:32<27:35:04, 27.49s/it]
 28%|██▊       | 1389/5000 [11:28:00<27:46:04, 27.68s/it]
 28%|██▊       | 1390/5000 [11:28:29<27:59:32, 27.91s/it]
 28%|██▊       | 1391/5000 [11:28:56<27:47:18, 27.72s/it]
 28%|██▊       | 1392/5000 [11:29:24<27:54:20, 27.84s/it]
 28%|██▊       | 1393/5000 [11:29:52<27:55:22, 27.87s/it]
 28%|██▊       | 1394/5000 [11:30:21<28:13:02, 28.17s/it]
 28%|██▊       | 1395/5000 [11:30:48<27:51:33, 27.82s/it]
 28%|██▊       | 1396/5000 [11:31:16<27:58:08, 27.94s/it]
 28%|██▊       | 1397/5000 [11:31:45<28:12:19, 28.18s/it]
 28%|██▊       | 1398/5000 [11:32:12<27:57:18, 27.94s/it]
 28%|██▊       | 1399/5000 [11:32:41<28:02:08, 28.03s/it]
 28%|██▊       | 1400/5000 [11:33:09<28:05:10, 28.09s/it]
                                                         
 28%|██▊       | 1400/5000 [11:33:09<28:05:10, 28.09s/it]
 28%|██▊       | 1401/5000 [11:33:36<27:53:26, 27.90s/it]
 28%|██▊       | 1402/5000 [11:34:05<28:02:49, 28.06s/it]
 28%|██▊       | 1403/5000 [11:34:33<28:06:52, 28.14s/it]
 28%|██▊       | 1404/5000 [11:35:01<27:55:53, 27.96s/it]
 28%|██▊       | 1405/5000 [11:35:29<28:01:31, 28.06s/it]
 28%|██▊       | 1406/5000 [11:35:57<28:04:24, 28.12s/it]
 28%|██▊       | 1407/5000 [11:36:24<27:51:24, 27.91s/it]
 28%|██▊       | 1408/5000 [11:36:52<27:49:08, 27.88s/it]
 28%|██▊       | 1409/5000 [11:37:21<28:01:06, 28.09s/it]
 28%|██▊       | 1410/5000 [11:37:49<27:53:02, 27.96s/it]
 28%|██▊       | 1411/5000 [11:38:18<28:12:46, 28.30s/it]
 28%|██▊       | 1412/5000 [11:38:45<27:53:21, 27.98s/it]
 28%|██▊       | 1413/5000 [11:39:12<27:34:11, 27.67s/it]
 28%|██▊       | 1414/5000 [11:39:41<28:00:46, 28.12s/it]
 28%|██▊       | 1415/5000 [11:40:08<27:41:46, 27.81s/it]
 28%|██▊       | 1416/5000 [11:40:35<27:26:29, 27.56s/it]
 28%|██▊       | 1417/5000 [11:41:03<27:28:07, 27.60s/it]
 28%|██▊       | 1418/5000 [11:41:31<27:36:52, 27.75s/it]
 28%|██▊       | 1419/5000 [11:41:58<27:30:02, 27.65s/it]
 28%|██▊       | 1420/5000 [11:42:27<27:40:21, 27.83s/it]
 28%|██▊       | 1421/5000 [11:42:55<27:45:15, 27.92s/it]
 28%|██▊       | 1422/5000 [11:43:22<27:34:57, 27.75s/it]
 28%|██▊       | 1423/5000 [11:43:50<27:44:26, 27.92s/it]
 28%|██▊       | 1424/5000 [11:44:18<27:42:26, 27.89s/it]
 28%|██▊       | 1425/5000 [11:44:46<27:43:38, 27.92s/it]
                                                         
 28%|██▊       | 1425/5000 [11:44:46<27:43:38, 27.92s/it]
 29%|██▊       | 1426/5000 [11:45:14<27:33:33, 27.76s/it]
 29%|██▊       | 1427/5000 [11:45:43<27:55:02, 28.13s/it]
 29%|██▊       | 1428/5000 [11:46:10<27:38:56, 27.87s/it]
 29%|██▊       | 1429/5000 [11:46:37<27:22:36, 27.60s/it]
 29%|██▊       | 1430/5000 [11:47:05<27:38:14, 27.87s/it]
 29%|██▊       | 1431/5000 [11:47:32<27:24:41, 27.65s/it]
 29%|██▊       | 1432/5000 [11:48:00<27:15:12, 27.50s/it]
 29%|██▊       | 1433/5000 [11:48:32<28:37:24, 28.89s/it]
 29%|██▊       | 1434/5000 [11:48:59<28:09:46, 28.43s/it]
 29%|██▊       | 1435/5000 [11:49:26<27:50:08, 28.11s/it]
 29%|██▊       | 1436/5000 [11:49:55<27:58:10, 28.25s/it]
 29%|██▊       | 1437/5000 [11:50:22<27:38:38, 27.93s/it]
 29%|██▉       | 1438/5000 [11:50:49<27:26:56, 27.74s/it]
 29%|██▉       | 1439/5000 [11:51:17<27:18:33, 27.61s/it]
 29%|██▉       | 1440/5000 [11:51:47<28:03:57, 28.38s/it]
 29%|██▉       | 1441/5000 [11:52:14<27:49:17, 28.14s/it]
 29%|██▉       | 1442/5000 [11:52:42<27:39:29, 27.98s/it]
 29%|██▉       | 1443/5000 [11:53:11<27:51:56, 28.20s/it]
 29%|██▉       | 1444/5000 [11:53:38<27:39:04, 27.99s/it]
 29%|██▉       | 1445/5000 [11:54:05<27:18:37, 27.66s/it]
 29%|██▉       | 1446/5000 [11:54:35<28:01:38, 28.39s/it]
 29%|██▉       | 1447/5000 [11:55:02<27:38:57, 28.02s/it]
 29%|██▉       | 1448/5000 [11:55:30<27:21:35, 27.73s/it]
 29%|██▉       | 1449/5000 [11:55:59<27:53:39, 28.28s/it]
 29%|██▉       | 1450/5000 [11:56:27<27:40:17, 28.06s/it]
                                                         
 29%|██▉       | 1450/5000 [11:56:27<27:40:17, 28.06s/it]
 29%|██▉       | 1451/5000 [11:56:54<27:26:28, 27.84s/it]
 29%|██▉       | 1452/5000 [11:57:23<27:48:44, 28.22s/it]
 29%|██▉       | 1453/5000 [11:57:50<27:30:27, 27.92s/it]
 29%|██▉       | 1454/5000 [11:58:16<26:58:29, 27.39s/it]
 29%|██▉       | 1455/5000 [11:58:46<27:28:33, 27.90s/it]
 29%|██▉       | 1456/5000 [11:59:13<27:13:41, 27.66s/it]
 29%|██▉       | 1457/5000 [11:59:40<27:05:51, 27.53s/it]
 29%|██▉       | 1458/5000 [12:00:09<27:32:16, 27.99s/it]
 29%|██▉       | 1459/5000 [12:00:36<27:17:05, 27.74s/it]
 29%|██▉       | 1460/5000 [12:01:04<27:11:50, 27.66s/it]
 29%|██▉       | 1461/5000 [12:01:32<27:30:47, 27.99s/it]
 29%|██▉       | 1462/5000 [12:02:00<27:19:06, 27.80s/it]
 29%|██▉       | 1463/5000 [12:02:27<27:11:13, 27.67s/it]
 29%|██▉       | 1464/5000 [12:02:57<27:43:10, 28.22s/it]
 29%|██▉       | 1465/5000 [12:03:24<27:28:50, 27.99s/it]
 29%|██▉       | 1466/5000 [12:03:51<27:17:06, 27.79s/it]
 29%|██▉       | 1467/5000 [12:04:21<27:47:06, 28.31s/it]
 29%|██▉       | 1468/5000 [12:04:35<23:32:33, 24.00s/it]
 29%|██▉       | 1469/5000 [12:04:45<19:37:47, 20.01s/it]
 29%|██▉       | 1470/5000 [12:04:56<16:57:46, 17.30s/it]
 29%|██▉       | 1471/5000 [12:05:07<15:02:59, 15.35s/it]{'loss': 0.0189, 'learning_rate': 8.171111111111113e-06, 'epoch': 8.0}
+{'loss': 0.0201, 'learning_rate': 8.115555555555557e-06, 'epoch': 8.01}
+{'loss': 0.0182, 'learning_rate': 8.06e-06, 'epoch': 8.01}
+{'loss': 0.013, 'learning_rate': 8.004444444444445e-06, 'epoch': 8.02}
+{'loss': 0.0148, 'learning_rate': 7.948888888888889e-06, 'epoch': 8.02}
+{'loss': 0.016, 'learning_rate': 7.893333333333335e-06, 'epoch': 8.03}
+
+
+
+
+
+
+
+
+
 29%|██▉       | 1472/5000 [12:06:37<36:46:56, 37.53s/it]
 29%|██▉       | 1473/5000 [12:07:05<34:05:45, 34.80s/it]
 29%|██▉       | 1474/5000 [12:07:33<32:09:36, 32.83s/it]
 30%|██▉       | 1475/5000 [12:08:01<30:35:51, 31.25s/it]
                                                         
 30%|██▉       | 1475/5000 [12:08:01<30:35:51, 31.25s/it]
 30%|██▉       | 1476/5000 [12:08:27<29:14:42, 29.88s/it]
 30%|██▉       | 1477/5000 [12:08:56<28:45:55, 29.39s/it]
 30%|██▉       | 1478/5000 [12:09:24<28:32:11, 29.17s/it]
 30%|██▉       | 1479/5000 [12:09:53<28:20:02, 28.97s/it]
 30%|██▉       | 1480/5000 [12:10:21<28:00:31, 28.65s/it]
 30%|██▉       | 1481/5000 [12:10:48<27:27:23, 28.09s/it]
 30%|██▉       | 1482/5000 [12:11:16<27:26:09, 28.08s/it]
 30%|██▉       | 1483/5000 [12:11:44<27:37:15, 28.27s/it]
 30%|██▉       | 1484/5000 [12:12:12<27:25:44, 28.08s/it]
 30%|██▉       | 1485/5000 [12:12:40<27:27:43, 28.13s/it]
 30%|██▉       | 1486/5000 [12:13:08<27:29:56, 28.17s/it]
 30%|██▉       | 1487/5000 [12:13:37<27:34:55, 28.27s/it]
 30%|██▉       | 1488/5000 [12:14:05<27:37:15, 28.31s/it]
 30%|██▉       | 1489/5000 [12:14:33<27:21:52, 28.06s/it]
 30%|██▉       | 1490/5000 [12:15:06<28:46:38, 29.52s/it]
 30%|██▉       | 1491/5000 [12:15:35<28:49:28, 29.57s/it]
 30%|██▉       | 1492/5000 [12:16:03<28:10:52, 28.92s/it]
 30%|██▉       | 1493/5000 [12:16:31<28:03:07, 28.80s/it]
 30%|██▉       | 1494/5000 [12:17:00<27:58:07, 28.72s/it]
 30%|██▉       | 1495/5000 [12:17:27<27:30:58, 28.26s/it]
 30%|██▉       | 1496/5000 [12:17:56<27:39:50, 28.42s/it]
 30%|██▉       | 1497/5000 [12:18:24<27:28:43, 28.24s/it]
 30%|██▉       | 1498/5000 [12:18:51<27:10:42, 27.94s/it]
 30%|██▉       | 1499/5000 [12:19:20<27:21:26, 28.13s/it]
 30%|███       | 1500/5000 [12:19:47<27:11:41, 27.97s/it]
                                                         
 30%|███       | 1500/5000 [12:19:47<27:11:41, 27.97s/it]
 30%|███       | 1501/5000 [12:20:15<27:01:40, 27.81s/it]
 30%|███       | 1502/5000 [12:20:43<27:12:26, 28.00s/it]
 30%|███       | 1503/5000 [12:21:11<27:04:51, 27.88s/it]
 30%|███       | 1504/5000 [12:21:38<26:58:32, 27.78s/it]
 30%|███       | 1505/5000 [12:22:07<27:24:25, 28.23s/it]
 30%|███       | 1506/5000 [12:22:35<27:07:09, 27.94s/it]
 30%|███       | 1507/5000 [12:23:02<26:52:11, 27.69s/it]
 30%|███       | 1508/5000 [12:23:30<27:08:51, 27.99s/it]
 30%|███       | 1509/5000 [12:23:59<27:13:40, 28.08s/it]
 30%|███       | 1510/5000 [12:24:26<27:06:58, 27.97s/it]
 30%|███       | 1511/5000 [12:24:55<27:21:39, 28.23s/it]
 30%|███       | 1512/5000 [12:25:24<27:23:35, 28.27s/it]
 30%|███       | 1513/5000 [12:25:50<26:56:58, 27.82s/it]
 30%|███       | 1514/5000 [12:26:19<27:08:41, 28.03s/it]
 30%|███       | 1515/5000 [12:26:47<27:14:18, 28.14s/it]
 30%|███       | 1516/5000 [12:27:14<26:55:57, 27.83s/it]
 30%|███       | 1517/5000 [12:27:43<27:07:38, 28.04s/it]
 30%|███       | 1518/5000 [12:28:11<27:02:39, 27.96s/it]
 30%|███       | 1519/5000 [12:28:38<26:54:32, 27.83s/it]
 30%|███       | 1520/5000 [12:29:06<26:44:12, 27.66s/it]
 30%|███       | 1521/5000 [12:29:33<26:39:53, 27.59s/it]
 30%|███       | 1522/5000 [12:30:03<27:21:36, 28.32s/it]
 30%|███       | 1523/5000 [12:30:31<27:07:24, 28.08s/it]
 30%|███       | 1524/5000 [12:31:04<28:34:47, 29.60s/it]
 30%|███       | 1525/5000 [12:31:31<27:52:56, 28.89s/it]
                                                         
 30%|███       | 1525/5000 [12:31:31<27:52:56, 28.89s/it]
 31%|███       | 1526/5000 [12:31:57<27:01:16, 28.00s/it]
 31%|███       | 1527/5000 [12:32:25<27:06:31, 28.10s/it]
 31%|███       | 1528/5000 [12:32:53<26:54:47, 27.91s/it]
 31%|███       | 1529/5000 [12:33:20<26:42:00, 27.69s/it]
 31%|███       | 1530/5000 [12:33:49<26:59:04, 28.00s/it]
 31%|███       | 1531/5000 [12:34:16<26:44:41, 27.75s/it]
 31%|███       | 1532/5000 [12:34:43<26:36:35, 27.62s/it]
 31%|███       | 1533/5000 [12:35:12<27:00:56, 28.05s/it]
 31%|███       | 1534/5000 [12:35:39<26:46:28, 27.81s/it]
 31%|███       | 1535/5000 [12:36:06<26:34:27, 27.61s/it]
 31%|███       | 1536/5000 [12:36:35<26:47:15, 27.84s/it]
 31%|███       | 1537/5000 [12:37:03<26:48:11, 27.86s/it]
 31%|███       | 1538/5000 [12:37:30<26:33:13, 27.61s/it]
 31%|███       | 1539/5000 [12:37:58<26:36:55, 27.68s/it]
 31%|███       | 1540/5000 [12:38:26<26:50:03, 27.92s/it]
 31%|███       | 1541/5000 [12:38:54<26:41:16, 27.78s/it]
 31%|███       | 1542/5000 [12:39:27<28:19:05, 29.48s/it]
 31%|███       | 1543/5000 [12:39:55<27:53:20, 29.04s/it]
 31%|███       | 1544/5000 [12:40:22<27:22:52, 28.52s/it]
 31%|███       | 1545/5000 [12:40:52<27:50:04, 29.00s/it]
 31%|███       | 1546/5000 [12:41:21<27:36:28, 28.77s/it]
 31%|███       | 1547/5000 [12:41:49<27:32:42, 28.72s/it]
 31%|███       | 1548/5000 [12:42:17<27:12:32, 28.38s/it]
 31%|███       | 1549/5000 [12:42:45<27:06:36, 28.28s/it]
 31%|███       | 1550/5000 [12:43:13<27:03:45, 28.24s/it]
                                                         
 31%|███       | 1550/5000 [12:43:13<27:03:45, 28.24s/it]
 31%|███       | 1551/5000 [12:43:40<26:46:01, 27.94s/it]
 31%|███       | 1552/5000 [12:44:09<26:51:27, 28.04s/it]
 31%|███       | 1553/5000 [12:44:36<26:46:02, 27.96s/it]
 31%|███       | 1554/5000 [12:45:04<26:33:26, 27.74s/it]
 31%|███       | 1555/5000 [12:45:31<26:30:35, 27.70s/it]
 31%|███       | 1556/5000 [12:46:00<26:48:06, 28.02s/it]
 31%|███       | 1557/5000 [12:46:27<26:37:19, 27.84s/it]
 31%|███       | 1558/5000 [12:46:56<26:44:03, 27.96s/it]
 31%|███       | 1559/5000 [12:47:24<26:44:24, 27.98s/it]
 31%|███       | 1560/5000 [12:47:51<26:32:17, 27.77s/it]
 31%|███       | 1561/5000 [12:48:19<26:35:55, 27.84s/it]
 31%|███       | 1562/5000 [12:48:47<26:45:43, 28.02s/it]
 31%|███▏      | 1563/5000 [12:49:15<26:36:00, 27.86s/it]
 31%|███▏      | 1564/5000 [12:49:43<26:47:23, 28.07s/it]
 31%|███▏      | 1565/5000 [12:50:11<26:30:26, 27.78s/it]
 31%|███▏      | 1566/5000 [12:50:37<26:06:30, 27.37s/it]
 31%|███▏      | 1567/5000 [12:51:05<26:15:14, 27.53s/it]
 31%|███▏      | 1568/5000 [12:51:33<26:32:55, 27.85s/it]
 31%|███▏      | 1569/5000 [12:52:01<26:22:45, 27.68s/it]
 31%|███▏      | 1570/5000 [12:52:29<26:29:00, 27.80s/it]
 31%|███▏      | 1571/5000 [12:52:57<26:42:31, 28.04s/it]
 31%|███▏      | 1572/5000 [12:53:26<26:44:39, 28.09s/it]
 31%|███▏      | 1573/5000 [12:53:53<26:33:34, 27.90s/it]
 31%|███▏      | 1574/5000 [12:54:23<26:59:56, 28.37s/it]
 32%|███▏      | 1575/5000 [12:54:51<26:59:30, 28.37s/it]
                                                         
 32%|███▏      | 1575/5000 [12:54:51<26:59:30, 28.37s/it]
 32%|███▏      | 1576/5000 [12:55:18<26:38:35, 28.01s/it]
 32%|███▏      | 1577/5000 [12:55:45<26:17:00, 27.64s/it]
 32%|███▏      | 1578/5000 [12:56:13<26:30:11, 27.88s/it]
 32%|███▏      | 1579/5000 [12:56:40<26:16:59, 27.66s/it]
 32%|███▏      | 1580/5000 [12:57:07<26:06:31, 27.48s/it]
 32%|███▏      | 1581/5000 [12:57:37<26:43:35, 28.14s/it]
 32%|███▏      | 1582/5000 [12:58:04<26:24:12, 27.81s/it]
 32%|███▏      | 1583/5000 [12:58:31<26:08:29, 27.54s/it]
 32%|███▏      | 1584/5000 [12:59:00<26:33:22, 27.99s/it]
 32%|███▏      | 1585/5000 [12:59:27<26:14:21, 27.66s/it]
 32%|███▏      | 1586/5000 [12:59:55<26:17:01, 27.72s/it]
 32%|███▏      | 1587/5000 [13:00:27<27:34:23, 29.08s/it]
 32%|███▏      | 1588/5000 [13:00:55<27:15:15, 28.76s/it]
 32%|███▏      | 1589/5000 [13:01:23<26:51:00, 28.34s/it]
 32%|███▏      | 1590/5000 [13:01:51<26:58:44, 28.48s/it]
 32%|███▏      | 1591/5000 [13:02:19<26:52:04, 28.37s/it]
 32%|███▏      | 1592/5000 [13:02:47<26:29:16, 27.98s/it]
 32%|███▏      | 1593/5000 [13:03:15<26:30:34, 28.01s/it]
 32%|███▏      | 1594/5000 [13:03:43<26:33:39, 28.07s/it]
 32%|███▏      | 1595/5000 [13:04:10<26:21:22, 27.87s/it]
 32%|███▏      | 1596/5000 [13:04:38<26:27:24, 27.98s/it]
 32%|███▏      | 1597/5000 [13:05:06<26:24:36, 27.94s/it]
 32%|███▏      | 1598/5000 [13:05:34<26:16:14, 27.80s/it]
 32%|███▏      | 1599/5000 [13:06:01<26:09:28, 27.69s/it]
 32%|███▏      | 1600/5000 [13:06:30<26:27:03, 28.01s/it]
                                                         
 32%|███▏      | 1600/5000 [13:06:30<26:27:03, 28.01s/it]
 32%|███▏      | 1601/5000 [13:06:56<26:00:03, 27.54s/it]
 32%|███▏      | 1602/5000 [13:07:24<25:56:00, 27.48s/it]
 32%|███▏      | 1603/5000 [13:07:53<26:30:08, 28.09s/it]
 32%|███▏      | 1604/5000 [13:08:21<26:18:15, 27.88s/it]
 32%|███▏      | 1605/5000 [13:08:47<25:57:29, 27.53s/it]
 32%|███▏      | 1606/5000 [13:09:17<26:27:58, 28.07s/it]
 32%|███▏      | 1607/5000 [13:09:44<26:12:10, 27.80s/it]
 32%|███▏      | 1608/5000 [13:10:11<26:03:09, 27.65s/it]
 32%|███▏      | 1609/5000 [13:10:39<26:07:40, 27.74s/it]
 32%|███▏      | 1610/5000 [13:11:07<26:05:11, 27.70s/it]
 32%|███▏      | 1611/5000 [13:11:34<25:53:08, 27.50s/it]
 32%|███▏      | 1612/5000 [13:12:02<26:08:36, 27.78s/it]
 32%|███▏      | 1613/5000 [13:12:30<26:17:08, 27.94s/it]
 32%|███▏      | 1614/5000 [13:12:58<26:06:32, 27.76s/it]
 32%|███▏      | 1615/5000 [13:13:26<26:15:27, 27.93s/it]
 32%|███▏      | 1616/5000 [13:13:54<26:07:22, 27.79s/it]
 32%|███▏      | 1617/5000 [13:14:21<25:58:13, 27.64s/it]
 32%|███▏      | 1618/5000 [13:14:49<26:12:17, 27.89s/it]
 32%|███▏      | 1619/5000 [13:15:18<26:19:24, 28.03s/it]
 32%|███▏      | 1620/5000 [13:15:45<26:06:14, 27.80s/it]
 32%|███▏      | 1621/5000 [13:16:15<26:36:44, 28.35s/it]
 32%|███▏      | 1622/5000 [13:16:42<26:20:53, 28.08s/it]
 32%|███▏      | 1623/5000 [13:17:09<26:08:41, 27.87s/it]
 32%|███▏      | 1624/5000 [13:17:39<26:33:00, 28.31s/it]
 32%|███▎      | 1625/5000 [13:18:05<26:04:53, 27.82s/it]
                                                         
 32%|███▎      | 1625/5000 [13:18:05<26:04:53, 27.82s/it]
 33%|███▎      | 1626/5000 [13:18:32<25:39:43, 27.38s/it]
 33%|███▎      | 1627/5000 [13:19:00<25:57:00, 27.70s/it]
 33%|███▎      | 1628/5000 [13:19:28<26:02:38, 27.80s/it]
 33%|███▎      | 1629/5000 [13:19:56<25:55:25, 27.68s/it]
 33%|███▎      | 1630/5000 [13:20:25<26:13:16, 28.01s/it]
 33%|███▎      | 1631/5000 [13:20:47<24:36:03, 26.29s/it]
 33%|███▎      | 1632/5000 [13:20:58<20:14:27, 21.64s/it]
 33%|███▎      | 1633/5000 [13:21:09<17:14:14, 18.43s/it]
 33%|███▎      | 1634/5000 [13:21:19<15:06:12, 16.15s/it]
 33%|███▎      | 1635/5000 [13:21:27<12:44:04, 13.62s/it]{'loss': 0.0138, 'learning_rate': 7.837777777777779e-06, 'epoch': 9.0}
+{'loss': 0.0132, 'learning_rate': 7.782222222222223e-06, 'epoch': 9.01}
+{'loss': 0.0128, 'learning_rate': 7.726666666666667e-06, 'epoch': 9.01}
+{'loss': 0.0096, 'learning_rate': 7.67111111111111e-06, 'epoch': 9.02}
+{'loss': 0.0085, 'learning_rate': 7.6155555555555564e-06, 'epoch': 9.02}
+{'loss': 0.0093, 'learning_rate': 7.5600000000000005e-06, 'epoch': 9.03}
+{'loss': 0.0108, 'learning_rate': 7.504444444444445e-06, 'epoch': 9.03}
+
+
+
+
+
+
+
+
+
 33%|███▎      | 1636/5000 [13:23:09<37:25:31, 40.05s/it]
 33%|███▎      | 1637/5000 [13:23:37<34:06:50, 36.52s/it]
 33%|███▎      | 1638/5000 [13:24:04<31:28:36, 33.71s/it]
 33%|███▎      | 1639/5000 [13:24:33<29:57:34, 32.09s/it]
 33%|███▎      | 1640/5000 [13:25:01<28:52:57, 30.95s/it]
 33%|███▎      | 1641/5000 [13:25:29<28:05:03, 30.10s/it]
 33%|███▎      | 1642/5000 [13:25:56<27:21:45, 29.33s/it]
 33%|███▎      | 1643/5000 [13:26:24<26:57:20, 28.91s/it]
 33%|███▎      | 1644/5000 [13:26:53<26:45:01, 28.70s/it]
 33%|███▎      | 1645/5000 [13:27:20<26:29:16, 28.42s/it]
 33%|███▎      | 1646/5000 [13:27:48<26:09:44, 28.08s/it]
 33%|███▎      | 1647/5000 [13:28:16<26:17:17, 28.22s/it]
 33%|███▎      | 1648/5000 [13:28:44<26:07:09, 28.05s/it]
 33%|███▎      | 1649/5000 [13:29:11<25:50:31, 27.76s/it]
 33%|███▎      | 1650/5000 [13:29:38<25:39:58, 27.58s/it]
                                                         
 33%|███▎      | 1650/5000 [13:29:38<25:39:58, 27.58s/it]
 33%|███▎      | 1651/5000 [13:30:05<25:34:33, 27.49s/it]
 33%|███▎      | 1652/5000 [13:30:33<25:44:02, 27.67s/it]
 33%|███▎      | 1653/5000 [13:31:01<25:33:44, 27.49s/it]
 33%|███▎      | 1654/5000 [13:31:30<25:57:38, 27.93s/it]
 33%|███▎      | 1655/5000 [13:31:57<25:45:05, 27.71s/it]
 33%|███▎      | 1656/5000 [13:32:24<25:32:51, 27.50s/it]
 33%|███▎      | 1657/5000 [13:32:52<25:47:19, 27.77s/it]
 33%|███▎      | 1658/5000 [13:33:20<25:52:06, 27.87s/it]
 33%|███▎      | 1659/5000 [13:33:47<25:39:52, 27.65s/it]
 33%|███▎      | 1660/5000 [13:34:16<25:53:40, 27.91s/it]
 33%|███▎      | 1661/5000 [13:34:43<25:42:34, 27.72s/it]
 33%|███▎      | 1662/5000 [13:35:10<25:32:21, 27.54s/it]
 33%|███▎      | 1663/5000 [13:35:38<25:30:18, 27.52s/it]
 33%|███▎      | 1664/5000 [13:36:05<25:22:15, 27.38s/it]
 33%|███▎      | 1665/5000 [13:36:31<25:09:22, 27.16s/it]
 33%|███▎      | 1666/5000 [13:37:01<25:44:47, 27.80s/it]
 33%|███▎      | 1667/5000 [13:37:28<25:32:31, 27.59s/it]
 33%|███▎      | 1668/5000 [13:37:54<25:08:43, 27.17s/it]
 33%|███▎      | 1669/5000 [13:38:23<25:31:37, 27.59s/it]
 33%|███▎      | 1670/5000 [13:38:50<25:21:33, 27.42s/it]
 33%|███▎      | 1671/5000 [13:39:16<25:09:14, 27.20s/it]
 33%|███▎      | 1672/5000 [13:39:45<25:38:28, 27.74s/it]
 33%|███▎      | 1673/5000 [13:40:12<25:15:14, 27.33s/it]
 33%|███▎      | 1674/5000 [13:40:38<25:03:16, 27.12s/it]
 34%|███▎      | 1675/5000 [13:41:06<25:19:32, 27.42s/it]
                                                         
 34%|███▎      | 1675/5000 [13:41:06<25:19:32, 27.42s/it]
 34%|███▎      | 1676/5000 [13:41:33<25:12:57, 27.31s/it]
 34%|███▎      | 1677/5000 [13:42:02<25:31:21, 27.65s/it]
 34%|███▎      | 1678/5000 [13:42:30<25:30:24, 27.64s/it]
 34%|███▎      | 1679/5000 [13:42:56<25:13:08, 27.34s/it]
 34%|███▎      | 1680/5000 [13:43:25<25:45:10, 27.92s/it]
 34%|███▎      | 1681/5000 [13:43:53<25:37:04, 27.79s/it]
 34%|███▎      | 1682/5000 [13:44:21<25:43:44, 27.92s/it]
 34%|███▎      | 1683/5000 [13:44:49<25:40:16, 27.86s/it]
 34%|███▎      | 1684/5000 [13:45:16<25:27:47, 27.64s/it]
 34%|███▎      | 1685/5000 [13:45:44<25:32:00, 27.73s/it]
 34%|███▎      | 1686/5000 [13:46:12<25:29:49, 27.70s/it]
 34%|███▎      | 1687/5000 [13:46:38<25:15:57, 27.45s/it]
 34%|███▍      | 1688/5000 [13:47:07<25:26:03, 27.65s/it]
 34%|███▍      | 1689/5000 [13:47:33<25:11:11, 27.38s/it]
 34%|███▍      | 1690/5000 [13:48:01<25:09:34, 27.36s/it]
 34%|███▍      | 1691/5000 [13:48:29<25:24:40, 27.65s/it]
 34%|███▍      | 1692/5000 [13:48:56<25:16:12, 27.50s/it]
 34%|███▍      | 1693/5000 [13:49:23<25:01:04, 27.23s/it]
 34%|███▍      | 1694/5000 [13:49:51<25:22:41, 27.64s/it]
 34%|███▍      | 1695/5000 [13:50:18<25:14:46, 27.50s/it]
 34%|███▍      | 1696/5000 [13:50:46<25:10:07, 27.42s/it]
 34%|███▍      | 1697/5000 [13:51:14<25:29:33, 27.78s/it]
 34%|███▍      | 1698/5000 [13:51:41<25:18:17, 27.59s/it]
 34%|███▍      | 1699/5000 [13:52:08<25:06:27, 27.38s/it]
 34%|███▍      | 1700/5000 [13:52:38<25:50:17, 28.19s/it]
                                                         
 34%|███▍      | 1700/5000 [13:52:38<25:50:17, 28.19s/it]
 34%|███▍      | 1701/5000 [13:53:04<25:13:16, 27.52s/it]
 34%|███▍      | 1702/5000 [13:53:33<25:30:11, 27.84s/it]
 34%|███▍      | 1703/5000 [13:54:01<25:37:22, 27.98s/it]
 34%|███▍      | 1704/5000 [13:54:29<25:27:40, 27.81s/it]
 34%|███▍      | 1705/5000 [13:54:57<25:42:19, 28.08s/it]
 34%|███▍      | 1706/5000 [13:55:25<25:33:51, 27.94s/it]
 34%|███▍      | 1707/5000 [13:55:53<25:33:44, 27.95s/it]
 34%|███▍      | 1708/5000 [13:56:21<25:37:52, 28.03s/it]
 34%|███▍      | 1709/5000 [13:56:48<25:20:10, 27.72s/it]
 34%|███▍      | 1710/5000 [13:57:16<25:28:39, 27.88s/it]
 34%|███▍      | 1711/5000 [13:57:44<25:21:54, 27.76s/it]
 34%|███▍      | 1712/5000 [13:58:12<25:18:23, 27.71s/it]
 34%|███▍      | 1713/5000 [13:58:40<25:30:41, 27.94s/it]
 34%|███▍      | 1714/5000 [13:59:08<25:29:34, 27.93s/it]
 34%|███▍      | 1715/5000 [13:59:35<25:15:18, 27.68s/it]
 34%|███▍      | 1716/5000 [14:00:05<25:47:17, 28.27s/it]
 34%|███▍      | 1717/5000 [14:00:34<25:57:15, 28.46s/it]
 34%|███▍      | 1718/5000 [14:01:01<25:41:12, 28.18s/it]
 34%|███▍      | 1719/5000 [14:01:30<25:45:47, 28.27s/it]
 34%|███▍      | 1720/5000 [14:01:57<25:36:59, 28.12s/it]
 34%|███▍      | 1721/5000 [14:02:30<26:45:27, 29.38s/it]
 34%|███▍      | 1722/5000 [14:02:57<26:07:19, 28.69s/it]
 34%|███▍      | 1723/5000 [14:03:29<27:06:41, 29.78s/it]
 34%|███▍      | 1724/5000 [14:03:58<26:45:27, 29.40s/it]
 34%|███▍      | 1725/5000 [14:04:25<26:15:06, 28.86s/it]
                                                         
 34%|███▍      | 1725/5000 [14:04:25<26:15:06, 28.86s/it]
 35%|███▍      | 1726/5000 [14:04:53<25:57:25, 28.54s/it]
 35%|███▍      | 1727/5000 [14:05:21<25:53:10, 28.47s/it]
 35%|███▍      | 1728/5000 [14:05:49<25:34:32, 28.14s/it]
 35%|███▍      | 1729/5000 [14:06:17<25:44:27, 28.33s/it]
 35%|███▍      | 1730/5000 [14:06:46<25:41:59, 28.29s/it]
 35%|███▍      | 1731/5000 [14:07:13<25:24:43, 27.99s/it]
 35%|███▍      | 1732/5000 [14:07:41<25:34:15, 28.17s/it]
 35%|███▍      | 1733/5000 [14:08:10<25:33:06, 28.16s/it]
 35%|███▍      | 1734/5000 [14:08:37<25:20:07, 27.93s/it]
 35%|███▍      | 1735/5000 [14:09:05<25:20:19, 27.94s/it]
 35%|███▍      | 1736/5000 [14:09:33<25:25:28, 28.04s/it]
 35%|███▍      | 1737/5000 [14:10:01<25:12:55, 27.82s/it]
 35%|███▍      | 1738/5000 [14:10:30<25:42:46, 28.38s/it]
 35%|███▍      | 1739/5000 [14:10:58<25:27:33, 28.11s/it]
 35%|███▍      | 1740/5000 [14:11:25<25:16:29, 27.91s/it]
 35%|███▍      | 1741/5000 [14:11:54<25:34:43, 28.26s/it]
 35%|███▍      | 1742/5000 [14:12:21<25:15:11, 27.90s/it]
 35%|███▍      | 1743/5000 [14:12:48<25:03:08, 27.69s/it]
 35%|███▍      | 1744/5000 [14:13:17<25:16:42, 27.95s/it]
 35%|███▍      | 1745/5000 [14:13:45<25:22:19, 28.06s/it]
 35%|███▍      | 1746/5000 [14:14:13<25:08:41, 27.82s/it]
 35%|███▍      | 1747/5000 [14:14:41<25:19:52, 28.03s/it]
 35%|███▍      | 1748/5000 [14:15:08<25:05:57, 27.79s/it]
 35%|███▍      | 1749/5000 [14:15:36<25:00:19, 27.69s/it]
 35%|███▌      | 1750/5000 [14:16:09<26:21:11, 29.19s/it]
                                                         
 35%|███▌      | 1750/5000 [14:16:09<26:21:11, 29.19s/it]
 35%|███▌      | 1751/5000 [14:16:39<26:42:55, 29.60s/it]
 35%|███▌      | 1752/5000 [14:17:07<26:22:37, 29.24s/it]
 35%|███▌      | 1753/5000 [14:17:34<25:45:51, 28.57s/it]
 35%|███▌      | 1754/5000 [14:18:03<25:46:47, 28.59s/it]
 35%|███▌      | 1755/5000 [14:18:31<25:26:59, 28.23s/it]
 35%|███▌      | 1756/5000 [14:18:58<25:07:45, 27.89s/it]
 35%|███▌      | 1757/5000 [14:19:27<25:31:09, 28.33s/it]
 35%|███▌      | 1758/5000 [14:19:54<25:15:10, 28.04s/it]
 35%|███▌      | 1759/5000 [14:20:21<24:59:00, 27.75s/it]
 35%|███▌      | 1760/5000 [14:20:51<25:21:43, 28.18s/it]
 35%|███▌      | 1761/5000 [14:21:18<25:06:13, 27.90s/it]
 35%|███▌      | 1762/5000 [14:21:45<24:53:38, 27.68s/it]
 35%|███▌      | 1763/5000 [14:22:19<26:32:47, 29.52s/it]
 35%|███▌      | 1764/5000 [14:22:46<25:54:14, 28.82s/it]
 35%|███▌      | 1765/5000 [14:23:13<25:23:46, 28.26s/it]
 35%|███▌      | 1766/5000 [14:23:40<25:07:10, 27.96s/it]
 35%|███▌      | 1767/5000 [14:24:09<25:19:26, 28.20s/it]
 35%|███▌      | 1768/5000 [14:24:36<25:06:20, 27.96s/it]
 35%|███▌      | 1769/5000 [14:25:03<24:50:11, 27.67s/it]
 35%|███▌      | 1770/5000 [14:25:33<25:17:01, 28.18s/it]
 35%|███▌      | 1771/5000 [14:26:00<24:59:39, 27.87s/it]
 35%|███▌      | 1772/5000 [14:26:27<24:53:26, 27.76s/it]
 35%|███▌      | 1773/5000 [14:26:56<25:13:41, 28.14s/it]
 35%|███▌      | 1774/5000 [14:27:24<24:58:53, 27.88s/it]
 36%|███▌      | 1775/5000 [14:27:51<24:48:36, 27.69s/it]
                                                         
 36%|███▌      | 1775/5000 [14:27:51<24:48:36, 27.69s/it]
 36%|███▌      | 1776/5000 [14:28:19<24:56:20, 27.85s/it]
 36%|███▌      | 1777/5000 [14:28:47<24:51:23, 27.76s/it]
 36%|███▌      | 1778/5000 [14:29:14<24:42:01, 27.60s/it]
 36%|███▌      | 1779/5000 [14:29:42<24:53:03, 27.81s/it]
 36%|███▌      | 1780/5000 [14:30:10<24:48:58, 27.74s/it]
 36%|███▌      | 1781/5000 [14:30:37<24:40:13, 27.59s/it]
 36%|███▌      | 1782/5000 [14:31:07<25:12:30, 28.20s/it]
 36%|███▌      | 1783/5000 [14:31:34<25:00:34, 27.99s/it]
 36%|███▌      | 1784/5000 [14:32:02<24:50:43, 27.81s/it]
 36%|███▌      | 1785/5000 [14:32:31<25:21:21, 28.39s/it]
 36%|███▌      | 1786/5000 [14:32:59<25:05:46, 28.11s/it]
 36%|███▌      | 1787/5000 [14:33:26<24:48:35, 27.80s/it]
 36%|███▌      | 1788/5000 [14:33:54<24:49:08, 27.82s/it]
 36%|███▌      | 1789/5000 [14:34:21<24:41:28, 27.68s/it]
 36%|███▌      | 1790/5000 [14:34:48<24:32:55, 27.53s/it]
 36%|███▌      | 1791/5000 [14:35:17<24:59:21, 28.03s/it]
 36%|███▌      | 1792/5000 [14:35:45<24:48:49, 27.85s/it]
 36%|███▌      | 1793/5000 [14:36:12<24:30:17, 27.51s/it]
 36%|███▌      | 1794/5000 [14:36:41<24:54:35, 27.97s/it]
 36%|███▌      | 1795/5000 [14:36:54<21:05:37, 23.69s/it]
 36%|███▌      | 1796/5000 [14:37:05<17:37:54, 19.81s/it]
 36%|███▌      | 1797/5000 [14:37:16<15:14:25, 17.13s/it]
 36%|███▌      | 1798/5000 [14:37:27<13:32:16, 15.22s/it]{'loss': 0.0086, 'learning_rate': 7.44888888888889e-06, 'epoch': 10.0}
+{'loss': 0.0085, 'learning_rate': 7.393333333333333e-06, 'epoch': 10.01}
+{'loss': 0.0087, 'learning_rate': 7.337777777777778e-06, 'epoch': 10.01}
+{'loss': 0.006, 'learning_rate': 7.282222222222222e-06, 'epoch': 10.02}
+{'loss': 0.0064, 'learning_rate': 7.226666666666667e-06, 'epoch': 10.02}
+{'loss': 0.0072, 'learning_rate': 7.171111111111112e-06, 'epoch': 10.03}
+
+
+
+
+
+
+
+
+
 36%|███▌      | 1799/5000 [14:39:04<35:31:12, 39.95s/it]
 36%|███▌      | 1800/5000 [14:39:33<32:23:23, 36.44s/it]
                                                         
 36%|███▌      | 1800/5000 [14:39:33<32:23:23, 36.44s/it]
 36%|███▌      | 1801/5000 [14:40:01<30:07:58, 33.91s/it]
 36%|███▌      | 1802/5000 [14:40:29<28:37:19, 32.22s/it]
 36%|███▌      | 1803/5000 [14:40:56<27:13:59, 30.67s/it]
 36%|███▌      | 1804/5000 [14:41:25<26:41:19, 30.06s/it]
 36%|███▌      | 1805/5000 [14:41:52<26:05:36, 29.40s/it]
 36%|███▌      | 1806/5000 [14:42:20<25:31:20, 28.77s/it]
 36%|███▌      | 1807/5000 [14:42:48<25:20:18, 28.57s/it]
 36%|███▌      | 1808/5000 [14:43:15<24:55:49, 28.12s/it]
 36%|███▌      | 1809/5000 [14:43:43<24:54:39, 28.10s/it]
 36%|███▌      | 1810/5000 [14:44:16<26:11:14, 29.55s/it]
 36%|███▌      | 1811/5000 [14:44:43<25:37:31, 28.93s/it]
 36%|███▌      | 1812/5000 [14:45:12<25:37:11, 28.93s/it]
 36%|███▋      | 1813/5000 [14:45:41<25:31:39, 28.84s/it]
 36%|███▋      | 1814/5000 [14:46:09<25:20:29, 28.63s/it]
 36%|███▋      | 1815/5000 [14:46:37<25:07:30, 28.40s/it]
 36%|███▋      | 1816/5000 [14:47:05<24:53:54, 28.15s/it]
 36%|███▋      | 1817/5000 [14:47:32<24:45:59, 28.01s/it]
 36%|███▋      | 1818/5000 [14:48:01<24:51:11, 28.12s/it]
 36%|███▋      | 1819/5000 [14:48:28<24:40:39, 27.93s/it]
 36%|███▋      | 1820/5000 [14:48:57<24:54:24, 28.20s/it]
 36%|███▋      | 1821/5000 [14:49:25<24:52:56, 28.18s/it]
 36%|███▋      | 1822/5000 [14:49:52<24:35:39, 27.86s/it]
 36%|███▋      | 1823/5000 [14:50:20<24:37:45, 27.91s/it]
 36%|███▋      | 1824/5000 [14:50:48<24:39:31, 27.95s/it]
 36%|███▋      | 1825/5000 [14:51:15<24:28:15, 27.75s/it]
                                                         
 36%|███▋      | 1825/5000 [14:51:16<24:28:15, 27.75s/it]
 37%|███▋      | 1826/5000 [14:51:44<24:45:45, 28.09s/it]
 37%|███▋      | 1827/5000 [14:52:13<24:48:43, 28.15s/it]
 37%|███▋      | 1828/5000 [14:52:40<24:32:10, 27.85s/it]
 37%|███▋      | 1829/5000 [14:53:09<24:45:40, 28.11s/it]
 37%|███▋      | 1830/5000 [14:53:35<24:20:58, 27.65s/it]
 37%|███▋      | 1831/5000 [14:54:02<24:12:02, 27.49s/it]
 37%|███▋      | 1832/5000 [14:54:31<24:38:40, 28.01s/it]
 37%|███▋      | 1833/5000 [14:54:58<24:14:57, 27.56s/it]
 37%|███▋      | 1834/5000 [14:55:25<24:03:56, 27.36s/it]
 37%|███▋      | 1835/5000 [14:55:53<24:10:50, 27.50s/it]
 37%|███▋      | 1836/5000 [14:56:20<24:14:11, 27.58s/it]
 37%|███▋      | 1837/5000 [14:56:48<24:09:05, 27.49s/it]
 37%|███▋      | 1838/5000 [14:57:16<24:17:08, 27.65s/it]
 37%|███▋      | 1839/5000 [14:57:44<24:25:02, 27.81s/it]
 37%|███▋      | 1840/5000 [14:58:11<24:18:52, 27.70s/it]
 37%|███▋      | 1841/5000 [14:58:39<24:19:25, 27.72s/it]
 37%|███▋      | 1842/5000 [14:59:07<24:26:37, 27.86s/it]
 37%|███▋      | 1843/5000 [14:59:34<24:02:28, 27.41s/it]
 37%|███▋      | 1844/5000 [15:00:02<24:08:51, 27.54s/it]
 37%|███▋      | 1845/5000 [15:00:29<24:12:56, 27.63s/it]
 37%|███▋      | 1846/5000 [15:00:59<24:39:11, 28.14s/it]
 37%|███▋      | 1847/5000 [15:01:25<24:13:52, 27.67s/it]
 37%|███▋      | 1848/5000 [15:01:52<24:03:48, 27.48s/it]
 37%|███▋      | 1849/5000 [15:02:21<24:26:37, 27.93s/it]
 37%|███▋      | 1850/5000 [15:02:48<24:08:51, 27.60s/it]
                                                         
 37%|███▋      | 1850/5000 [15:02:48<24:08:51, 27.60s/it]
 37%|███▋      | 1851/5000 [15:03:16<24:18:32, 27.79s/it]
 37%|███▋      | 1852/5000 [15:03:44<24:17:52, 27.79s/it]
 37%|███▋      | 1853/5000 [15:04:11<23:58:03, 27.42s/it]
 37%|███▋      | 1854/5000 [15:04:42<25:01:11, 28.63s/it]
 37%|███▋      | 1855/5000 [15:05:10<24:48:46, 28.40s/it]
 37%|███▋      | 1856/5000 [15:05:37<24:30:29, 28.06s/it]
 37%|███▋      | 1857/5000 [15:06:07<24:49:58, 28.44s/it]
 37%|███▋      | 1858/5000 [15:06:34<24:31:44, 28.10s/it]
 37%|███▋      | 1859/5000 [15:07:01<24:12:28, 27.75s/it]
 37%|███▋      | 1860/5000 [15:07:30<24:31:36, 28.12s/it]
 37%|███▋      | 1861/5000 [15:07:57<24:20:04, 27.91s/it]
 37%|███▋      | 1862/5000 [15:08:25<24:10:41, 27.74s/it]
 37%|███▋      | 1863/5000 [15:08:53<24:25:45, 28.04s/it]
 37%|███▋      | 1864/5000 [15:09:21<24:25:20, 28.04s/it]
 37%|███▋      | 1865/5000 [15:09:49<24:11:48, 27.79s/it]
 37%|███▋      | 1866/5000 [15:10:16<24:07:50, 27.72s/it]
 37%|███▋      | 1867/5000 [15:10:44<24:08:44, 27.74s/it]
 37%|███▋      | 1868/5000 [15:11:11<23:50:26, 27.40s/it]
 37%|███▋      | 1869/5000 [15:11:40<24:19:09, 27.96s/it]
 37%|███▋      | 1870/5000 [15:12:07<24:05:39, 27.71s/it]
 37%|███▋      | 1871/5000 [15:12:34<23:57:34, 27.57s/it]
 37%|███▋      | 1872/5000 [15:13:03<24:15:18, 27.92s/it]
 37%|███▋      | 1873/5000 [15:13:35<25:25:58, 29.28s/it]
 37%|███▋      | 1874/5000 [15:14:04<25:09:04, 28.97s/it]
 38%|███▊      | 1875/5000 [15:14:31<24:36:48, 28.35s/it]
                                                         
 38%|███▊      | 1875/5000 [15:14:31<24:36:48, 28.35s/it]
 38%|███▊      | 1876/5000 [15:14:59<24:32:58, 28.29s/it]
 38%|███▊      | 1877/5000 [15:15:27<24:31:43, 28.28s/it]
 38%|███▊      | 1878/5000 [15:15:54<24:18:21, 28.03s/it]
 38%|███▊      | 1879/5000 [15:16:27<25:23:08, 29.28s/it]
 38%|███▊      | 1880/5000 [15:16:55<25:07:16, 28.99s/it]
 38%|███▊      | 1881/5000 [15:17:22<24:42:22, 28.52s/it]
 38%|███▊      | 1882/5000 [15:17:51<24:43:18, 28.54s/it]
 38%|███▊      | 1883/5000 [15:18:19<24:43:19, 28.55s/it]
 38%|███▊      | 1884/5000 [15:18:47<24:27:07, 28.25s/it]
 38%|��██▊      | 1885/5000 [15:19:15<24:29:34, 28.31s/it]
 38%|███▊      | 1886/5000 [15:19:44<24:30:25, 28.33s/it]
 38%|███▊      | 1887/5000 [15:20:11<24:14:05, 28.03s/it]
 38%|███▊      | 1888/5000 [15:20:39<24:18:14, 28.12s/it]
 38%|███▊      | 1889/5000 [15:21:07<24:14:50, 28.06s/it]
 38%|███▊      | 1890/5000 [15:21:35<24:06:48, 27.91s/it]
 38%|███▊      | 1891/5000 [15:22:04<24:17:57, 28.14s/it]
 38%|███▊      | 1892/5000 [15:22:32<24:26:48, 28.32s/it]
 38%|███▊      | 1893/5000 [15:23:00<24:09:43, 28.00s/it]
 38%|███▊      | 1894/5000 [15:23:28<24:15:09, 28.11s/it]
 38%|███▊      | 1895/5000 [15:23:57<24:22:03, 28.25s/it]
 38%|███▊      | 1896/5000 [15:24:24<24:04:27, 27.92s/it]
 38%|███▊      | 1897/5000 [15:24:52<24:07:46, 27.99s/it]
 38%|███▊      | 1898/5000 [15:25:20<24:12:01, 28.09s/it]
 38%|███▊      | 1899/5000 [15:25:48<24:12:41, 28.11s/it]
 38%|███▊      | 1900/5000 [15:26:16<24:03:23, 27.94s/it]
                                                         
 38%|███▊      | 1900/5000 [15:26:16<24:03:23, 27.94s/it]
 38%|███▊      | 1901/5000 [15:26:44<24:03:15, 27.94s/it]
 38%|███▊      | 1902/5000 [15:27:12<24:05:04, 27.99s/it]
 38%|███▊      | 1903/5000 [15:27:39<23:54:36, 27.79s/it]
 38%|███▊      | 1904/5000 [15:28:06<23:42:02, 27.56s/it]
 38%|███▊      | 1905/5000 [15:28:38<24:41:45, 28.73s/it]
 38%|███▊      | 1906/5000 [15:29:04<24:08:42, 28.09s/it]
 38%|███▊      | 1907/5000 [15:29:32<23:55:44, 27.85s/it]
 38%|███▊      | 1908/5000 [15:30:01<24:18:45, 28.31s/it]
 38%|███▊      | 1909/5000 [15:30:28<24:02:18, 28.00s/it]
 38%|███▊      | 1910/5000 [15:30:55<23:47:19, 27.72s/it]
 38%|███▊      | 1911/5000 [15:31:30<25:29:01, 29.70s/it]
 38%|███▊      | 1912/5000 [15:31:57<24:54:20, 29.04s/it]
 38%|███▊      | 1913/5000 [15:32:26<24:46:17, 28.89s/it]
 38%|███▊      | 1914/5000 [15:32:54<24:40:57, 28.79s/it]
 38%|███▊      | 1915/5000 [15:33:23<24:33:13, 28.65s/it]
 38%|███▊      | 1916/5000 [15:33:50<24:14:18, 28.29s/it]
 38%|███▊      | 1917/5000 [15:34:18<24:12:28, 28.27s/it]
 38%|███▊      | 1918/5000 [15:34:47<24:11:54, 28.27s/it]
 38%|███▊      | 1919/5000 [15:35:14<23:57:00, 27.98s/it]
 38%|███▊      | 1920/5000 [15:35:42<24:04:11, 28.13s/it]
 38%|███▊      | 1921/5000 [15:36:11<24:07:01, 28.20s/it]
 38%|███▊      | 1922/5000 [15:36:38<23:55:02, 27.97s/it]
 38%|███▊      | 1923/5000 [15:37:07<24:04:12, 28.16s/it]
 38%|███▊      | 1924/5000 [15:37:35<24:05:39, 28.20s/it]
 38%|███▊      | 1925/5000 [15:38:02<23:49:05, 27.88s/it]
                                                         
 38%|███▊      | 1925/5000 [15:38:02<23:49:05, 27.88s/it]
 39%|███▊      | 1926/5000 [15:38:30<23:41:10, 27.74s/it]
 39%|███▊      | 1927/5000 [15:38:58<23:57:47, 28.07s/it]
 39%|███▊      | 1928/5000 [15:39:26<23:43:48, 27.81s/it]
 39%|███▊      | 1929/5000 [15:39:53<23:36:07, 27.67s/it]
 39%|███▊      | 1930/5000 [15:40:22<23:57:46, 28.10s/it]
 39%|███▊      | 1931/5000 [15:40:50<23:48:12, 27.92s/it]
 39%|███▊      | 1932/5000 [15:41:17<23:36:02, 27.69s/it]
 39%|███▊      | 1933/5000 [15:41:46<23:56:01, 28.09s/it]
 39%|███▊      | 1934/5000 [15:42:13<23:42:50, 27.84s/it]
 39%|███▊      | 1935/5000 [15:42:40<23:24:27, 27.49s/it]
 39%|███▊      | 1936/5000 [15:43:08<23:35:31, 27.72s/it]
 39%|███▊      | 1937/5000 [15:43:36<23:42:08, 27.86s/it]
 39%|███▉      | 1938/5000 [15:44:04<23:35:47, 27.74s/it]
 39%|███▉      | 1939/5000 [15:44:32<23:42:19, 27.88s/it]
 39%|███▉      | 1940/5000 [15:45:00<23:43:14, 27.91s/it]
 39%|███▉      | 1941/5000 [15:45:27<23:30:02, 27.66s/it]
 39%|███▉      | 1942/5000 [15:45:55<23:42:36, 27.91s/it]
 39%|███▉      | 1943/5000 [15:46:24<23:48:55, 28.05s/it]
 39%|███▉      | 1944/5000 [15:46:51<23:40:41, 27.89s/it]
 39%|███▉      | 1945/5000 [15:47:20<23:50:14, 28.09s/it]
 39%|███▉      | 1946/5000 [15:47:48<23:52:55, 28.15s/it]
 39%|███▉      | 1947/5000 [15:48:15<23:31:55, 27.75s/it]
 39%|███▉      | 1948/5000 [15:48:47<24:44:37, 29.19s/it]
 39%|███▉      | 1949/5000 [15:49:15<24:14:04, 28.60s/it]
 39%|███▉      | 1950/5000 [15:49:42<23:58:06, 28.29s/it]
                                                         
 39%|███▉      | 1950/5000 [15:49:42<23:58:06, 28.29s/it]
 39%|███▉      | 1951/5000 [15:50:10<23:56:41, 28.27s/it]
 39%|███▉      | 1952/5000 [15:50:37<23:27:30, 27.71s/it]
 39%|███▉      | 1953/5000 [15:51:04<23:20:49, 27.58s/it]
 39%|███▉      | 1954/5000 [15:51:33<23:33:34, 27.84s/it]
 39%|███▉      | 1955/5000 [15:52:00<23:31:36, 27.81s/it]
 39%|███▉      | 1956/5000 [15:52:28<23:27:02, 27.73s/it]
 39%|███▉      | 1957/5000 [15:52:57<23:48:47, 28.17s/it]
 39%|███▉      | 1958/5000 [15:53:19<22:17:09, 26.37s/it]
 39%|███▉      | 1959/5000 [15:53:30<18:20:33, 21.71s/it]
 39%|███▉      | 1960/5000 [15:53:41<15:34:09, 18.44s/it]
 39%|███▉      | 1961/5000 [15:53:52<13:37:41, 16.14s/it]
 39%|███▉      | 1962/5000 [15:54:00<11:31:58, 13.67s/it]{'loss': 0.0063, 'learning_rate': 7.115555555555557e-06, 'epoch': 11.0}
+{'loss': 0.006, 'learning_rate': 7.06e-06, 'epoch': 11.01}
+{'loss': 0.006, 'learning_rate': 7.004444444444445e-06, 'epoch': 11.01}
+{'loss': 0.0053, 'learning_rate': 6.948888888888889e-06, 'epoch': 11.02}
+{'loss': 0.0044, 'learning_rate': 6.893333333333334e-06, 'epoch': 11.02}
+{'loss': 0.0045, 'learning_rate': 6.837777777777779e-06, 'epoch': 11.03}
+{'loss': 0.0051, 'learning_rate': 6.782222222222222e-06, 'epoch': 11.03}
+
+
+
+
+
+
+
+
+
 39%|███▉      | 1963/5000 [15:55:49<35:48:04, 42.44s/it]
 39%|███▉      | 1964/5000 [15:56:18<32:14:09, 38.22s/it]
 39%|███▉      | 1965/5000 [15:56:45<29:34:02, 35.07s/it]
 39%|███▉      | 1966/5000 [15:57:13<27:49:21, 33.01s/it]
 39%|███▉      | 1967/5000 [15:57:42<26:33:25, 31.52s/it]
 39%|███▉      | 1968/5000 [15:58:09<25:31:00, 30.30s/it]
 39%|███▉      | 1969/5000 [15:58:35<24:25:58, 29.02s/it]
 39%|███▉      | 1970/5000 [15:59:02<23:58:24, 28.48s/it]
 39%|███▉      | 1971/5000 [15:59:30<23:54:33, 28.42s/it]
 39%|███▉      | 1972/5000 [15:59:59<23:56:40, 28.47s/it]
 39%|███▉      | 1973/5000 [16:00:26<23:39:18, 28.13s/it]
 39%|███▉      | 1974/5000 [16:00:55<23:40:04, 28.16s/it]
 40%|███▉      | 1975/5000 [16:01:23<23:43:18, 28.23s/it]
                                                         
 40%|███▉      | 1975/5000 [16:01:23<23:43:18, 28.23s/it]
 40%|███▉      | 1976/5000 [16:01:50<23:18:54, 27.76s/it]
 40%|███▉      | 1977/5000 [16:02:15<22:48:42, 27.17s/it]
 40%|███▉      | 1978/5000 [16:02:44<23:09:22, 27.59s/it]
 40%|███▉      | 1979/5000 [16:03:12<23:16:07, 27.73s/it]
 40%|███▉      | 1980/5000 [16:03:40<23:11:08, 27.64s/it]
 40%|███▉      | 1981/5000 [16:04:10<23:51:34, 28.45s/it]
 40%|███▉      | 1982/5000 [16:04:37<23:28:46, 28.01s/it]
 40%|███▉      | 1983/5000 [16:05:03<22:58:47, 27.42s/it]
 40%|███▉      | 1984/5000 [16:05:31<23:10:05, 27.65s/it]
 40%|███▉      | 1985/5000 [16:05:59<23:17:24, 27.81s/it]
 40%|███▉      | 1986/5000 [16:06:27<23:10:17, 27.68s/it]
 40%|███▉      | 1987/5000 [16:06:55<23:13:45, 27.76s/it]
 40%|███▉      | 1988/5000 [16:07:21<22:49:43, 27.29s/it]
 40%|███▉      | 1989/5000 [16:07:47<22:30:02, 26.90s/it]
 40%|███▉      | 1990/5000 [16:08:17<23:12:12, 27.75s/it]
 40%|███▉      | 1991/5000 [16:08:44<23:01:36, 27.55s/it]
 40%|███▉      | 1992/5000 [16:09:11<22:56:53, 27.46s/it]
 40%|███▉      | 1993/5000 [16:09:44<24:18:04, 29.09s/it]
 40%|███▉      | 1994/5000 [16:10:11<23:50:07, 28.55s/it]
 40%|███▉      | 1995/5000 [16:10:38<23:32:28, 28.20s/it]
 40%|███▉      | 1996/5000 [16:11:07<23:34:32, 28.25s/it]
 40%|███▉      | 1997/5000 [16:11:34<23:14:15, 27.86s/it]
 40%|███▉      | 1998/5000 [16:12:01<22:57:58, 27.54s/it]
 40%|███▉      | 1999/5000 [16:12:28<22:50:43, 27.41s/it]
 40%|████      | 2000/5000 [16:12:55<22:45:51, 27.32s/it]
                                                         
 40%|████      | 2000/5000 [16:12:55<22:45:51, 27.32s/it][INFO|trainer.py:3138] 2023-05-08 02:47:05,817 >> ***** Running Evaluation *****
+[INFO|trainer.py:3142] 2023-05-08 02:47:05,817 >>   Num examples: Unknown
+[INFO|trainer.py:3143] 2023-05-08 02:47:05,817 >>   Batch size = 64
+{'loss': 0.005, 'learning_rate': 6.726666666666667e-06, 'epoch': 12.0}
+{'loss': 0.0042, 'learning_rate': 6.671111111111112e-06, 'epoch': 12.01}
+
+
+
+[INFO|trainer_utils.py:693] 2023-05-08 02:47:16,424 >> The following columns in the evaluation set don't have a corresponding argument in `WhisperForConditionalGeneration.forward` and have been ignored: input_length. If input_length are not expected by `WhisperForConditionalGeneration.forward`,  you can safely ignore this message.
+
                                                         
 40%|████      | 2000/5000 [16:48:47<22:45:51, 27.32s/it][INFO|trainer.py:2877] 2023-05-08 03:22:58,551 >> Saving model checkpoint to ./checkpoint-2000
+[INFO|configuration_utils.py:458] 2023-05-08 03:22:58,556 >> Configuration saved in ./checkpoint-2000/config.json
+[INFO|configuration_utils.py:364] 2023-05-08 03:22:58,560 >> Configuration saved in ./checkpoint-2000/generation_config.json
+[INFO|modeling_utils.py:1855] 2023-05-08 03:23:01,997 >> Model weights saved in ./checkpoint-2000/pytorch_model.bin
+[INFO|feature_extraction_utils.py:369] 2023-05-08 03:23:02,003 >> Feature extractor saved in ./checkpoint-2000/preprocessor_config.json
+[INFO|feature_extraction_utils.py:369] 2023-05-08 03:23:12,574 >> Feature extractor saved in ./preprocessor_config.json

wandb/run-20230507_103405-9zf5xxpu/files/output.log

@@ -1077,3 +1077,1071 @@ Reading metadata...: 23329it [00:00, 27403.88it/s]
 [INFO|feature_extraction_utils.py:369] 2023-05-07 19:00:20,758 >> Feature extractor saved in ./checkpoint-1000/preprocessor_config.json
 [INFO|feature_extraction_utils.py:369] 2023-05-07 19:00:30,115 >> Feature extractor saved in ./preprocessor_config.json
 Adding files tracked by Git LFS: ['wandb/run-20230506_113337-ysywp688/run-ysywp688.wandb', 'wandb/run-20230507_103405-9zf5xxpu/run-9zf5xxpu.wandb']. This may take a bit of time if the files are large.
+05/07/2023 19:00:40 - WARNING - huggingface_hub.repository - Adding files tracked by Git LFS: ['wandb/run-20230506_113337-ysywp688/run-ysywp688.wandb', 'wandb/run-20230507_103405-9zf5xxpu/run-9zf5xxpu.wandb']. This may take a bit of time if the files are large.
+/home/local/QCRI/dizham/miniconda3/envs/whisper/lib/python3.9/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
+  warnings.warn('Was asked to gather along dimension 0, but all '
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 20%|██        | 1024/5000 [8:37:52<30:54:36, 27.99s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 21%|██        | 1049/5000 [8:49:36<30:47:50, 28.06s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 21%|██▏       | 1074/5000 [9:01:12<30:10:29, 27.67s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 22%|██▏       | 1099/5000 [9:12:51<29:28:49, 27.21s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 22%|██▎       | 1125/5000 [9:24:55<30:10:55, 28.04s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 23%|██▎       | 1144/5000 [9:32:41<16:23:31, 15.30s/it]
+
+Reading metadata...: 23270it [00:01, 13633.81it/s]
+
+
+
+
+
+
+
+ 23%|██▎       | 1150/5000 [9:36:28<31:22:23, 29.34s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 24%|██▎       | 1175/5000 [9:48:12<29:43:46, 27.98s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 24%|██▍       | 1199/5000 [9:59:22<29:47:08, 28.21s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 24%|██▍       | 1224/5000 [10:11:01<29:19:05, 27.95s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 25%|██▌       | 1250/5000 [10:23:19<28:54:41, 27.75s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 26%|██▌       | 1275/5000 [10:35:05<28:48:22, 27.84s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 26%|██▌       | 1299/5000 [10:46:17<28:38:13, 27.86s/it]
+
+
+
+
+
+
+
+
+ 26%|██▌       | 1308/5000 [10:49:12<14:02:21, 13.69s/it]
+
+Reading metadata...: 10438it [00:00, 26165.49it/s]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 26%|██▋       | 1324/5000 [10:57:48<28:34:52, 27.99s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 27%|██▋       | 1349/5000 [11:09:24<28:12:24, 27.81s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 27%|██▋       | 1374/5000 [11:21:05<27:52:56, 27.68s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 28%|██▊       | 1399/5000 [11:32:41<28:02:08, 28.03s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 28%|██▊       | 1425/5000 [11:44:46<27:43:38, 27.92s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 29%|██▉       | 1450/5000 [11:56:27<27:40:17, 28.06s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 29%|██▉       | 1471/5000 [12:05:07<15:02:59, 15.35s/it]
+
+Reading metadata...: 10438it [00:00, 26428.08it/s]
+
+
+
+ 29%|██▉       | 1474/5000 [12:07:33<32:09:36, 32.83s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 30%|██▉       | 1499/5000 [12:19:20<27:21:26, 28.13s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 30%|███       | 1525/5000 [12:31:31<27:52:56, 28.89s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 31%|███       | 1549/5000 [12:42:45<27:06:36, 28.28s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 31%|███▏      | 1574/5000 [12:54:23<26:59:56, 28.37s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 32%|███▏      | 1599/5000 [13:06:01<26:09:28, 27.69s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 32%|███▏      | 1624/5000 [13:17:39<26:33:00, 28.31s/it]
+
+
+
+
+
+
+
+
+
+
+ 33%|███▎      | 1635/5000 [13:21:27<12:44:04, 13.62s/it]
+
+Reading metadata...: 10438it [00:00, 27803.36it/s]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 33%|███▎      | 1650/5000 [13:29:38<25:39:58, 27.58s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 34%|███▎      | 1675/5000 [13:41:06<25:19:32, 27.42s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 34%|███▍      | 1700/5000 [13:52:38<25:50:17, 28.19s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 34%|███▍      | 1725/5000 [14:04:25<26:15:06, 28.86s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 35%|███▍      | 1749/5000 [14:15:36<25:00:19, 27.69s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 36%|███▌      | 1775/5000 [14:27:51<24:48:36, 27.69s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Reading metadata...: 28043it [00:01, 15924.27it/s]22s/it]
+Reading metadata...: 10438it [00:00, 12100.29it/s]
+
+
+ 36%|███▌      | 1800/5000 [14:39:33<32:23:23, 36.44s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 36%|███▋      | 1825/5000 [14:51:16<24:28:15, 27.75s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 37%|███▋      | 1850/5000 [15:02:48<24:08:51, 27.60s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 37%|███▋      | 1874/5000 [15:14:04<25:09:04, 28.97s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 38%|███▊      | 1900/5000 [15:26:16<24:03:23, 27.94s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 38%|███▊      | 1924/5000 [15:37:35<24:05:39, 28.20s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 39%|███▉      | 1950/5000 [15:49:42<23:58:06, 28.29s/it]
+
+
+
+
+
+
+
+
+
+
+
+ 39%|███▉      | 1962/5000 [15:54:00<11:31:58, 13.67s/it]
+
+Reading metadata...: 10438it [00:00, 22408.00it/s]
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 40%|███▉      | 1975/5000 [16:01:23<23:43:18, 28.23s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 40%|████      | 2000/5000 [16:12:55<22:45:51, 27.32s/it][INFO|trainer.py:3138] 2023-05-08 02:47:05,817 >> ***** Running Evaluation *****
+[INFO|trainer.py:3142] 2023-05-08 02:47:05,817 >>   Num examples: Unknown
+[INFO|trainer.py:3143] 2023-05-08 02:47:05,817 >>   Batch size = 64
+{'loss': 0.0042, 'learning_rate': 6.671111111111112e-06, 'epoch': 12.01}
+Reading metadata...: 10440it [00:00, 30685.17it/s]
+
+[INFO|trainer_utils.py:693] 2023-05-08 02:47:16,424 >> The following columns in the evaluation set don't have a corresponding argument in `WhisperForConditionalGeneration.forward` and have been ignored: input_length. If input_length are not expected by `WhisperForConditionalGeneration.forward`,  you can safely ignore this message.
+{'eval_loss': 0.5698409676551819, 'eval_wer': 55.900000000000006, 'eval_runtime': 2152.7247, 'eval_samples_per_second': 4.85, 'eval_steps_per_second': 0.076, 'epoch': 12.01}
+ 40%|████      | 2000/5000 [16:48:47<22:45:51, 27.32s/it][INFO|trainer.py:2877] 2023-05-08 03:22:58,551 >> Saving model checkpoint to ./checkpoint-2000
+[INFO|configuration_utils.py:458] 2023-05-08 03:22:58,556 >> Configuration saved in ./checkpoint-2000/config.json
+[INFO|configuration_utils.py:364] 2023-05-08 03:22:58,560 >> Configuration saved in ./checkpoint-2000/generation_config.json
+[INFO|modeling_utils.py:1855] 2023-05-08 03:23:01,997 >> Model weights saved in ./checkpoint-2000/pytorch_model.bin
+[INFO|feature_extraction_utils.py:369] 2023-05-08 03:23:02,003 >> Feature extractor saved in ./checkpoint-2000/preprocessor_config.json

wandb/run-20230507_103405-9zf5xxpu/files/wandb-summary.json

@@ -1 +1 @@
-{"eval/loss": 0.43405279517173767, "eval/wer": 54.25600000000001, "eval/runtime": 2248.2056, "eval/samples_per_second": 4.644, "eval/steps_per_second": 0.073, "_timestamp": 1683475217.3777876, "_runtime": 30371.4238345623, "_step": 40}
+{"eval/loss": 0.5698409676551819, "eval/wer": 55.900000000000006, "eval/runtime": 2152.7247, "eval/samples_per_second": 4.85, "eval/steps_per_second": 0.076, "_timestamp": 1683505378.5423114, "_runtime": 60532.58835840225, "_step": 81}

wandb/run-20230507_103405-9zf5xxpu/run-9zf5xxpu.wandb

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e77d49c926099ac2b0f7a7e3b4e00ce2e9407dda9d529e412727a2fb9b212be2
-size 2787409
+oid sha256:0f427f677d5083d56399114c2fe48a9a9e499435e0ec1b3b9234b20f01c0c562
+size 5571854